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I.  INTRODUCTION 

A.  GENERAL  OVERVIEW 

A  Marine  Aviation  Logistics  Squadron  (MALS)  provides  intermediate 
maintenance  and  supply  support  for  flight-line  squadrons.  A  MALS  might  support  flight- 
line  squadrons  that  all  operate  the  same  type/model/series  (T/M/S)  aircraft  or  squadrons 
with  different  T/M/S  aircraft.  Each  T/M/S  has  its  own  support  network  within  the  greater 
aviation  logistics  (AVLOG)  community. 

Determining  the  level  or  quality  of  support  that  a  MALS  is  providing  its  flight¬ 
line  squadrons  is  quite  difficult.  This  is  because  a  MALS  performs  so  many  different 
types  of  support  functions  that  knowing  what  should  be  measured  can  be  unclear.  Each  of 
these  support  functions  also  consists  of  many  other  sub-functions,  cooperating  to  produce 
a  final  output,  whether  that  output  is  a  part  or  a  service. 

The  civilian  sector  utilizes  many  different  methods  of  measuring  performance  in 
its  organizations  and  several  are  universally  agreed  upon  as  best  business  practices. 
Applying  these  performance  measurement  systems  directly  to  a  MALS  is  difficult  for 
many  reasons  but  primarily  because  financial  measurements  are  a  large  part  of  these 
civilian  measurement  systems.  Furthermore,  the  supplier-customer  relationship  in  the 
civilian  sector  is  not  identical  to  the  MALS-flight  line  relationship  in  AVLOG. 

This  thesis  addresses  these  issues,  and  answers  three  primary  questions  and  one 
secondary  question. 

1.  Primary  Questions 

•  Of  the  metrics  that  MALS  currently  employs,  which  measures  (or 
combination  thereof)  most  appropriately  reflect  the  level  of  support  the 
MALS  provides  the  flight-line? 

•  Of  the  metrics  that  MALS  currently  employs,  which  measures  most 
appropriately  incentivize  the  most  beneficial  behavior  to  support  the 
flight-line? 

•  Are  there  metrics  that  MALS  does  not  employ  that  should  be  adopted? 
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2.  Secondary  Question 

•  To  what  extent  can  standardized  performance  measurements  be  used  to 
measure  flight-line  support  performance  between  various  MALS? 

Many  of  the  logistic  metrics  currently  in  use  measure  their  respective  processes 
sufficiently.  These  individual  metrics,  however,  are  unable  to  function  as  a  performance 
measurement  for  the  MALS  as  a  whole.  This  is  because  they  do  not  sufficiently  measure 
the  output  of  the  MALS  as  a  whole.  A  proper  performance  measurement  system  for  a 
logistics  organization  will  capture  the  output  of  that  organization,  where  the  output  is  a 
function  of  all  the  sub-functions  within  the  organization.  While  a  MALS  has  numerous 
and  varied  sub-functions  and  responsibilities,  only  those  that  directly  contribute  to  the 
goal  of  the  MALS  are  to  be  included  in  the  performance  measurement  system. 

Supply  Effectiveness  is  found  to  be,  of  the  metrics  currently  used,  the  metric  that 
most  appropriately  reflects  the  level  of  support  the  MALS  provides  the  flight-line. 
Repairable  Physical  Buffer  Status  Red  and  TRR-Supply  are  currently  used  metrics  that 
incentivize  the  most  appropriate  behavior  to  support  the  flight-line.  Supply  Chain 
Response  Time  is  an  individual  metric  that  should  be  adopted  by  the  MALS  that  is  not 
currently  used.  A  weighted  performance  metric  composed  of  three  sub-metrics  (Physical 
Buffer  Status  Red  of  Repairables,  Supply  Chain  Response  Time,  and  Supply 
Effectiveness)  should  be  adopted  as  an  overall  performance  system  that  unifies  the 
command  towards  a  common  goal.  Lastly,  these  metrics  ought  to  be  aircraft 
type/model/series  specific  in  order  to  make  comparisons  of  performance  between  various 
MALS. 
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II.  BACKGROUND 


A.  INTRODUCTION 

This  chapter  focuses  on  the  current  structure,  organization,  and  measurement 
systems  used  in  a  Marine  Aviation  Logistics  Squadron  (MALS).  The  organizational 
structure  of  a  MALS  discussed  here  is  specific  to  the  current  structure  of  MALS  in 
garrison  settings.  The  systems  discussed  are  the  various  systems  accessible  to  a  MALS 
and  are  used  to  measure,  report,  analyze,  or  otherwise  manage  data.  Any  logistics  or 
performance  metrics  used  within  a  MALS  are  generally  computed  by  these  systems. 

B.  ORGANIZATIONAL  STRUCTURE  OF  A  MARINE  AVIATION 

LOGISTICS  SQUADRON 

A  MALS  exists  to  provide  aviation  logistics  (AVLOG)  support  of  intermediate 
level  maintenance  and  supply  functions  to  flight  line  squadrons.  A  typical  MALS  consists 
of  five  departments:  Maintenance,  Supply,  Avionics,  Ordnance,  and  Headquarters.  Due 
to  their  similarities  and  relationship,  the  Maintenance,  Avionics,  and  Ordnance 
departments  are  collectively  referred  to  as  the  Maintenance  department  in  this  thesis. 

A  MALS  may  support  flight  line  squadrons  that  all  fly  the  same  type/model/series 
(T/M/S)  aircraft  but,  more  commonly,  a  MALS  will  support  multiple  squadrons  with 
different  T/M/S.  This  is  depicted  in  Figure  1.  MALS-24  in  Hawaii,  for  example,  supports 
one  CH-53E  squadron,  one  H-l  squadron  that  flies  AH-1W  and  UH-1Y  aircraft,  four 
Navy  P-3  squadrons  (part  of  Commander,  Patrol,  and  Reconnaissance  Wing  Two),  and 
one  H-60  squadron.  Each  squadron’s  aircraft  T/M/S  is  different  from  the  others  (with  the 
exception  of  the  P-3  squadrons).  In  practical  terms,  this  means  the  supply  and 
maintenance  support  required  from  the  supporting  MALS  becomes  more  cumbersome 
and  complex  with  each  different  T/M/S  supported. 
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TACTICAL 

SQUADRON 


MARINE  AVIATION 
LOGISTICS  SQUADRON 


O-  Level 


Figure  1.  MALS  Support  Organization  Example.  Source:  United  States  Marine  Corps 

(2012). 


The  Aircraft  Maintenance  Officer,  typically  an  0-4  limited  duty  officer,  leads  the 
Maintenance  Department.  According  to  the  Marine  Corps  Warfighting  Publication 
3021.2,  “the  AVLOG  functions  of  the  MALS  maintenance  department  include  aircraft, 
avionics,  ground  support  equipment  (support  equipment)  maintenance,  flight  equipment, 
cryogenics,  and  maintenance  data  collection  and  analysis”  (United  States  Marine  Corps, 
2012,  p.  3-1).  Each  of  these  departments  is  broken  down  into  divisions  with  each 
focusing  on  one  major  function.  The  Power  Plants  400  division,  for  example,  primarily 
focuses  on  engine  overhauls.  The  Ground  Support  Equipment  800  division  repairs  and 
maintains  large  support  equipment  for  use  by  the  MALS  and  flight  line  squadrons. 

The  Aviation  Supply  Officer,  typically  an  0-4  unrestricted  officer,  leads  the 
Supply  Department.  The  supply  department  “executes  all  storage,  inventory,  condition, 
and  management  functions  of  Navy-provided,  aeronautical-related  materiel”  (United 
States  Marine  Corps,  2012,  p.  3-4).  For  example,  the  Supply  Response  Division  (SRD)  is 
responsible  “for  the  initial  screening  and  technical  research  of  all  requisitions  ordered 
through  NALCOMIS”  (United  States  Marine  Corps,  2012,  p.  3-4). 
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C.  MALSP  MODERNIZATION 

A  MALS  is  part  of  the  larger  Marine  Aviation  Logistics  Support  Program 
(MALSP).  The  MALSP  framework  was  recently  reconstructed  and  retitled  MALSP 
Modernization,  or  MALSP  MOD.  MALSP  MOD  is  “an  expeditionary  demand-pull 
logistics  capability  solution  that  aligns  with  national  defense  and  security  strategies,  as 
well  as:  the  long  war  concept,  Marine  Corps  vision  and  strategy  2025,  the  maritime 
strategy,  the  aviation  plan,  and  the  aviation  logistics  (AVLOG)  strategy”  (Headquarters 
Marine  Corps,  2016,  p.  1). 

Historically,  MALSP  functioned  as  a  push  supply  system.  A  push  supply  system 
supports  the  end  user  by  pushing  resources  forward  to  the  user,  regardless  of  whether 
requisitions  for  those  resources  have  been  made.  This  involves  a  large  footprint  of  supply 
inventory  and  repair  capability  in  forward  deployed  areas.  An  entire  MALS  would  be 
moved  to  forward  areas,  providing  intermediate  level  support  to  other  deployed 
squadrons. 

MALSP  MOD  seeks  to  create  a  pull  supply  system.  A  pull  system’s  action  is 
dictated  by  the  demand  signal  produced  by  the  customer.  The  paradigms  on  which 
MALSP  and  MALSP  MOD  are  based  are  quite  different.  Rather  than  a  supplier  pushing 
parts  to  fill  shelf  space,  parts  are  pulled  by  the  customer  to  fill  a  demand.  Rather  than  a 
large  supply  footprint  forward  deployed,  a  parent  MALS  (PMALS)  supports  deployed 
squadrons  from  garrison  locations,  as  depicted  in  Figure  2.  These  PMALS  send 
requisitioned  parts  forward  according  to  a  flight  line  squadron’s  demand  signal.  The 
logistics  chain  is  established  along  various  nodes  between  the  PMALS  and  the  deployed 
squadrons.  Navy  T-AVB  ships,  Main  Operating  Bases  (MOBs),  and  Forward  Operating 
Bases  (FOBs)  are  used  to  link  the  PMALS  with  the  squadrons. 
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Figure  2.  MALSP  II  Nodal  Network.  Source:  Borrelli  (201 1). 


A  key  aspect  of  MALSP  MOD  is  the  updated  technology  suite  titled  Marine 
Aviation  Logistics  Enterprise  Information  Technology  (MAL-EIT)  which  will 
accompany  the  new  program.  With  the  intent  of  being  a  net-centric  logistics  chain,  the 
next  generation  buffer  management  system  (NGBMS),  expeditionary  pack-up  kit 
(EPUK),  and  logistics  planning  tool  (LPT)  have  been  created  to  better  support  all  MALS 
and  squadrons  in  Marine  aviation.  These  software  packages,  in  combination  with  the 
nodal  logistics  chain,  “provide  more  responsive,  scalable,  and  flexible  solutions  through 
properly- sized  spares  packages  while  achieving  near  real  time  global  visibility” 
(Headquarters  Marine  Corps,  2016,  p.  2). 

D.  BUFFER  MANAGEMENT  TOOL 

The  current  buffer  management  tool  (BMT)  is  the  primary  software  package 
designed  for  use  by  the  maintenance  and  supply  departments  to  prioritize  repair  and 
replenishment  requirements.  Division  officers  in  the  MALS  are  instructed  to  “use  BMT 
reports  (I-level)  to  monitor  daily  workload  and  assign  priorities  to  ensure  efficient 
movement  of  components  through  assigned  work  centers”  (Department  of  the  Navy, 
2012,  p.  3-92).  The  BMT  is  described  in  the  Naval  Aviation  Maintenance  Program 
(NAMP)  as  follows: 
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[The  BMT]  pulls  data  from  NALCOMIS  and  R-Supply  databases  to 
generate  integrated  reports  that  provide  focus  on  what  work  has  priority 
for  repair  and  replenishment  of  buffers  in  a  Time  Domain.  Monitoring 
consumable  and  repairable  components,  from  a  time  domain  perspective, 
gives  supervisors  the  information  they  require  to  ensure  they  are  working 
on  the  items  that  are  a  priority  to  the  customer  producing  aircraft  ready  for 
tasking  and  not  sub-optimizing  resources  by  working  on  lower  priority 
items.  (Department  of  the  Navy,  2012,  p.  A-9) 

The  BMT  provides  maintenance  and  supply  managers  with  the  requisite 
information  to  prioritize  their  work.  It  can  produce  a  wealth  of  reports  useful  to 
maintenance  managers,  such  as  the  expeditious  repair  (EXREP)  report,  work  load  report, 
and  back  log  report.  Other  reports  help  managers  identify  what  the  next  EXREP  might  be 
due  to  their  buffer  status  as  well  as  identify  how  the  MALS  may  reduce  costs  caused  by 
being  unable  to  repair  components  in-house  (i.e.,  causing  a  repairable  to  be  considered 
beyond  the  capability  of  maintenance,  or  BCM).  The  supply  BMT  “provides  activities 
with  a  means  of  monitoring  and  managing  the  time  to  reliably  replenish  (TRR)  of  all 
components  under  TRR  Management.  The  TRR  value  of  a  requisitioned  part  starts  from 
the  time  a  requisition  is  ordered  and  ends  when  the  requisition  is  completed”  (NAVAIR 
6.7.2. 1,  2013,  p.  G-5). 

The  BMT  prioritizes  work  according  to  its  buffer  status.  The  buffer  status  is 
determined  by  the  item’s  TRR.  The  TRR  is  defined  as  “the  total  time  it  takes  once  a  part 
is  pulled  from  the  Supply  shelf  until  it  is  back  on  the  Supply  shelf  ready  for  issue” 
(NAVAIR  6.7.2. 1,  2013,  p.  G-30).  To  compute  the  TRR,  the  times  to  fill  historical 
requisitions  of  a  particular  item  are  measured.  The  90th  percentile  of  those  times  is  then 
applied  as  the  TRR  for  that  part  (Seagren,  2013). 

Each  requisition  has  a  color  code  assigned  based  on  its  buffer  status.  The  buffer 
status  has  both  a  time  and  physical  dimension.  In  regard  to  time,  the  code  refers  to  how 
long  it  will  take  to  repair  the  item  and  get  it  back  on  the  supply  officer’s  shelf  ready  for 
issue.  Physical  buffer  status  is  characterized  by  how  many  items  are  on  the  shelf.  The 
order  of  prioritization  of  work  orders,  in  descending  order  of  importance,  is  EXREPs, 
red,  yellow,  green,  and  black.  Figure  3  is  an  example  workload  priority  decision  matrix 
demonstrating  the  interaction  of  the  time  and  physical  components  of  TRR. 
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TRR  “Rules  of  Engagement” 

EXREP  always  gets  top  priority  I 

Work “Red"  TRR  first  (first in-first  served) 

I  |  Work ‘’Yellow"  TRR  when  at  stoppage  on  red 

1  1  Work  “Green"  TRR  after  stoppage  on  yeflow  and  red 

Work  “Black"  into  rotation  after  red-yellow -green  ( Prioritize  TRR  Black  items  with  physical  buffer  needs  i.e.  SUPPO's  shelf) 


Excess 

III 

III 

III 

III 

Green 

11/  III 

II 

II 

II 

Yellow 

I 

I 

I 

Green 

TRR 

Yellow 

TRR 

Black 

TRR 

PARTS 
BUFFER 
/  (Physical 
Inventory) 


SUPPO’S 

SHELF 


TIME  (TRR)  BUFFER 


Figure  3.  Example  BMT  Workload  Priority  Decision  Matrix. 

Source:  NAVAIR  6.7.2. 1  (2013). 


The  Maintenance  and  Supply  departments  are  able  to  extract  a  multitude  of 
reports  analyzing  their  current  workload.  Examples  of  the  broad  view  of  Supply 
Department’s  workload  in  terms  of  Requisition  TRR,  Consumable  TRR,  Repairable 
TRR,  and  Physical  Buffer  Status  are  provided  in  Figures  4,  5,  6,  and  7. 


Supply  Buffer  Management  Tool 

Last  Upbad Date/Ttme;  5/18/2010  5b3b3  PM 


Supply  Department  Management  Menu 

Requisition  TRR  |  TRR  Analysis  1  Consumable  TRR  |  Repairable  TRR  |  AWP  Analysis  |  Retrograde  |  Physical  Buffer  |  Reports  ]  Query  ] 


Overall  TRR  Processing  Summary 


Jj 


Requisitions 

9100 


Green 

Yellow 

Red  |  Black  | 

Current  TRR  Status 

0 

203 

806  8091 

0% 

9%  89% 

Percent  in  Zone: 

2% 

Consumable  TRR  Processing  Summary 


Requl  iltloni 
8413 


|  Green 

Yellow  | 

Current  TRR  Statui: 

0 

I 

Percent  in  “one: 

0%  j 

_ 1 

806  7404 

10%  88% 


Repaii  able  TRR  Processing  Summary 


Requl  iltlon  i 
687 


Current  TRR  Statu «: 


Percent  in  zone: 


Green  Yellow  |  Red  Bach 


Requisitions  TRR  Black  and  Buffer  Status  Red:  0 
SO  Retrograde  TRR  Black  and  Buffer  Status  Red:  55 

MAFs  Meeting  50%  Repair/BCM  Threshold:  39 

Achieved  90  Percentile  TRR  Exceeds  Max  Design  Off-Station  TRR  by  3645  Days 


Figure  4.  Supply  Requisition  TRR  Summary.  Adapted  from  NAVAIR  (201 1). 
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Figure  5.  Supply  Consumable  TRR  Summary.  Adapted  from  NAVAIR  (201 1). 


Supply  Buffer  Management  Tool 

ianr  Upfaad  Date/Time:  5/18/2010  5 $3 $3  PM _ 


Supply  Department  Management  Menu 

Requisition  TRR  |  JRR  Analysis  |  Consumable  TRR  Repairable  TRR  J  AWP  Analysis  |  Retrograde  |  Physical  Buffer  |  Reports  ]  Query  | 


0  0  687 

0 %  |9  0%  100% 


Repair «il»le  DTO  TRR  Piocessiiuj  Summary 

Requliltom 

687 

|  Green 

fellow 

|  Ped  |  Bach  | 

Current  TP  P  Statu*: 

G 

G  687 

Percent  in  zone: 

M  j 

19 

0%  100% 

|  Repair  able  Stock  TRR  Pi  ocessincj  Summary  | 

Requliltloni 

1  Green 

yellow 

|  Ped  |  Blach  | 

Current  TP  R~5t3tuT""H 

0 

G  G 

0 

Percent  In  Zone: 

0%  0% 

View  Repairable  Stock  TRR  Processing  Distribution:  h|  View  Repairable  DTO  TRR  Processing  Distribution: 

View  Repairable  Stock  Frequency  Distribution:  ■]  View  Repairable  DTO  Frequency  Distribution:  ■] 


Total  ActiveMAFS  Inducted: 

971 

AcliveMAFS  Exceeding  Design  SI: 

92 

9.5% 

ActiveMAFS  Exceeding  Design  S2: 

44 

4.5% 

Figure  6.  Supply  Repairable  TRR  Summary. 

Adapted  from  NAVAIR  (201 1). 
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Supply  Buffer  Management  Tool 


Last  Uphad  Date/Tme:  5/18/2010 5 $3 $3  PM 


Supply  Department  Management  Menu 

Requisition  TRR  |  TRR  Analysis  1  ^Consumable  TRR  J  Repairable  TRR  1  AWP  Analysis  [  Retrograde  Physical  Buffer  |  Reports  1  Query  | 


Physical  Buffer  Summary 


Line  Items 

Excess 

Green 

Yellow 

Red  | 

11909 

Current  Status: 

6762 

3258 

858 

1031 

Percent  in  Zone: 

57% 

27% 

7% 

9% 

Red  Buffer  Status/Replensihment 

Buffer  Status  Red  With  Stock  Due:  0 

Buffer  Status  Red  With  SO  Retrograde:  43 

Buffer  Sizing  Summary 

Totals  Site  Line  Items:  19,911 

Sized  Design  BSz  >0:  11,909 

Sized- Design  OH  and  BSz  0:  8,002 

Total  Sized:  19,911  100.00% 

Supported:  19,911  100.00% 

Transitional  TR  R  Set :  1 9 .9 1 1 


Current  Allowance  vs  Design  BSz 

Cument  Allowance  >  Design  BSz:  0  $0.00 

Current  Allowance  <  Design  BSz:  0  $0.00 

Delta  Saving/Spendings:  $0.00 

Excess.Deficiency  to  Design  BSz 

Excess  of  Design  BSz  On- Hand:  6,796  $33,419,075.49 

Excess  of  Design  BSz  On- Order:  0  $0.00 

Deficient  to  Design  BSz:  2,537  $18,102,315.35 


Figure  7.  Supply  Physical  Buffer  Summary.  Adapted  from  NAVAIR  (201 1). 


The  Maintenance  Department’s  reports  are  formatted  similarly.  The  Department 
Summary  examines  backlog  and  physical  buffer  impact  of  inductions,  as  seen  in  Figure 
8.  Figure  9  is  an  example  Awaiting  Parts  (AWP)  report  that  summarizes  how  AWP 
MAFs  are  affecting  the  department’s  workload. 


Maintenance  Buffer  Management  Tool 


Las*  Uphad  Date/Time:  5/18/2010  5 $3 $3  PM 


Department  Management  Menu 

Department  Summary  |  TRR  Analysis  |  AWP  Impact  |  AWP  TRR  Analysis  |  Demand  |  Reports  \  Query  | 


Department  Induction  Backlog  Summary 

under  TP  R  Rooming 

Total 

TDi 

umlzed 

Cemand 

Total 

Cemand 

Non-  A  IP 
DIFM 

Inducted 

Maintenance 

AWP 

TDi 

971 

767 

204 

21 

110 

1 

1082 

99 

Physical  Buffei  Impact 

MATS  Inducted:  971 

MAFS  Physical  Buffer  Status  RED:  76  7.8% 

Line  Items  Physical  Buffer  Status  RED:  43 


Department  TRR  Piocessing  Summary 


Inductions 

971 


Green 

Yellow 

Red 

|  Black  | 

Current  TRR  Status: 

0 

33 

184 

774 

Percent  in  Zone: 

0% 

3% 

17% 

80% 

tlon-WIP  Physical  Buffers 

End  Items:  16 

DEF  to  Design  BSz:  11 


MAFs  Meeting  50%  Repair/B CM  Threshold:  39 
SO  Retrograde  TRR  Black  and  Buffer  Status  Red:  55 

ActiveMAF  90  Percentile  TRR  Exceeds  Max  Design  On-Station  TRR  by  602  Days 


Figure  8.  Maintenance  Department  Summary.  Adapted  from  NAVAIR  (201 1). 
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Maintenance  Buffer  Management  Tool 

Ljst  Uphid  Date/Time:  S/1S/2010 S.-03.-03  PM 


Department  Management  Menu 

Department  Summary  |  TRR  Analysis  AWP  Impact  |  AWP  TRR  Analysis  |  Demand  |  Reports  |  Query  | 


AWP  Impact  Summary 

MAFS  Inducted:  971  MAFS  Black:  774 

AWP  MAPS:  204  AWP  Black:  182 

Percent  AWP:  21.0%  AWP  Black  Pet:  23.5% 


Figure  9.  Maintenance  AWP  Impact  Summary. 

Adapted  from  NAVAIR  (201 1). 


E.  NALCOMIS 

The  Naval  Aviation  Logistics  Command  Management  Information  System 
(NALCOMIS)  is  the  software  program  that  manages  maintenance  activity  in  both 
Organizational  and  Intermediate  level  maintenance.  The  NALCOMIS  system  has  two 
different  versions,  one  optimized  for  the  Intermediate  level  (I-Level)  maintenance  and 
one  optimized  for  the  Organizational  level  (O-Level)  maintenance  (primarily 
incorporating  maintenance  action  forms  linked  to  specific  flight  line  aircraft). 

NALCOMIS  is  a  tool  intended  to  help  maintenance  managers  prioritize  and 
manage  their  workload.  There  are  two  primary  reports  that  are  built  into  NALCOMIS  at 
the  I-Level.  They  are  the  Daily  Production  Report  and  the  Production  Report. 

The  NAMP  states  that  the  Daily  Production  Report  “should  be  run  and  distributed 
on  a  daily  basis.  The  Daily  Production  Report  Part  2  provides  a  count  of  all  maintenance 
actions  accomplished  from  the  begin  date/time  to  the  end  date/time  as  selected  by  the 
user”  (Department  of  the  Navy,  2012,  p.  14-19).  This  Daily  Production  Report  is  an 
overview  of  maintenance  actions  that  have  been  completed.  Managers  can  use  this 
information  to  examine  the  current  status  of  any  Maintenance  Action  Form  (MAF)  for 
their  work  center  or  division. 
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The  NAMP  states  that  the  Production  Report  should  also  be  run  on  a  daily  basis. 
This  report  is  more  detailed,  providing  more  detail  about  work  stoppages  due  to  awaiting 
parts  (AWP)  or  if  a  part  is  deemed  beyond  the  capability  of  maintenance  that  the  work 
center  can  provide  (BCM).  This  report  “provides  a  snapshot  of  each  work  center’s 
production  with  the  number  of  components  and  status  in  the  maintenance  cycle 
(including  OH,  AWP,  RFI,  BCM,  and  backlog)  within  a  specified  begin  and  end  date.  A 
percentage  of  RFI/BCM  rate  is  computed,  showing  work  load  production  by  work 
center”  (Department  of  the  Navy,  2012,  p.  14-19). 

F.  R-SUPPLY 

R-Supply  is  Aviation  Supply’s  primary  software  program  that  provides  online 
inventory,  logistics,  and  financial  management  tools  to  the  Aviation  Supply  Department. 
A  wealth  of  data  is  contained  in  this  program.  While  various  measurements  can  be 
obtained  on  every  part  and  requisition  in  the  supply  system,  the  performance  metrics 
most  used  by  the  Aviation  Supply  Department  are  Net  Supply  Effectiveness  and  Gross 
Supply  Effectiveness. 

Net  Supply  Effectiveness  is  the  proportion  of  requisitions  filled  of  items  that  are 
carried  on  the  supply  shelf  (Seagren,  2013).  If  a  requisition  is  placed  for  an  item  that  is 
not  carried  on  the  Supply  Officer’s  shelf  then  it  is  not  considered  in  this  equation.  Gross 
Supply  Effectiveness,  however,  includes  those  items  that  are  not  carried  on  the  supply 
shelf  and  is  thus  the  proportion  of  requisitions  filled  to  total  requisitions  (2013). 

R-Supply  produces  two  reports  that  contain  these  metrics  as  well  as  others:  the 
Depth  Effectiveness  report  and  the  Logistics  Replenishment  (LOGREP)  Supply 
Effectiveness  report.  The  Depth  Effectiveness  “produces  a  summary  of  the  total  customer 
demands,  total  issues,  and  the  effectiveness  expressed  in  a  percentage”  (NAVSUP  P-732, 
2005,  p.  48).  The  LOGREP  Supply  Effectiveness  report  “produces  a  report  that  measures 
the  supplying  activity’s  ability  to  fill  customers’  requisitions  and  referrals  (frequencies). 
The  report  summarizes  the  total  number  of  customer  demands,  total  issues,  and  the 
effectiveness  expressed  as  a  percentage  by  cognizance  symbol  within  MIC”  (NAVSUP 
P-732,  2005,  p.  48). 
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G.  SUPPLY  RESPONSE  METRICS 

The  NAMP  describes  several  basic  concepts  and  guidelines  for  I-Level 
maintenance  in  regard  to  material  management.  These  include  having  standardized 
requisitioning  procedures,  maintaining  positive  control  of  all  accountable  material, 
maximizing  use  of  personnel  and  material  resources,  and  ensuring  supply  response  to 
material  demands  is  optimum  (Department  of  the  Navy,  2012).  Ensuring  supply  response 
to  material  demands  is  particularly  important  in  the  discussion  of  performance  metrics. 

The  NAMP  dictates  the  response  time  goals  the  Supply  Department  should  strive 
to  achieve,  determined  by  the  priority  of  the  part  requisitioned.  Priorities  1,  2,  and  3  are 
given  to  all  requisitions  with  Priority  1  being  the  most  important.  It  is  assigned  to  parts 
that  are  required  for  maintenance  on  an  aircraft  that  is  non-mission  capable.  Specifically, 
the  absence  of  these  parts  renders  an  aircraft  non-mission  capable.  The  response  time 
goals  for  these  aircraft  are  as  follows: 


Table  1.  Supply  Response  Time  Goals 


Issue  Priority 
Group 

Priority 

Designator 

Processing 

Time 

1 

1-3 

1  Hour 

2 

4-8 

2  Hours 

3 

9-15 

24  Hours 

It  is  primarily  the  Aviation  Supply  Officer’s  responsibility  to  ensure  response 
times  are  being  measured  and  reduced  (Department  of  the  Navy,  2012,  p.  9-17).  Each 
priority  group  has  a  goal  processing  time  in  which  the  requisition  is  to  be  filled  and 
delivered  to  the  squadron. 

The  NAMP  states  that  one  of  the  primary  goals  of  the  maintenance  and  supply 
personnel  at  all  echelons  is  to  reduce  response  time.  It  goes  on  to  state  that  “the 
[turnaround  time]  of  repairables  must  be  improved  through  better  control  and  reporting 
procedures”  (Department  of  the  Navy,  2012,  p.  5-28).  To  monitor  this  performance,  a 
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supply  response  time  (SRT)  metric  is  taken  that  indicates  what  percentage  of  requisitions 
are  delivered  within  their  target  time. 

H.  AIRSPEED 

The  Naval  Aviation  Enterprise  (NAE)  officially  implemented  a  program  called 
AIRSpeed  in  pursuit  of  increasing  fleet  readiness  in  a  cost-wise  manner.  The  overarching 
methodologies  of  AIRSpeed  are  the  Theory  of  Constraints,  Lean  Manufacturing,  and  Six 
Sigma.  AIRSpeed  is  Naval  Aviation’s  catalyst  for  continuous  process  improvement  (CPI) 
within  aircraft  maintenance  and  aviation  supply  functions.  It  “provides  a  structured 
approach  to  plan,  train,  integrate,  sustain,  and  monitor  best  business  practices  across  the 
NAE.  Functions  include  benchmarking,  analysis,  innovation,  progress  assessment, 
communications,  and  incorporation  of  best  practices  to  maximize  overall  benefits” 
(Department  of  the  Navy,  2012,  p.  1-7).  The  NAMP  identifies  six  fundamental  principles 
of  the  AIRSpeed  program  (p.  1-7): 

•  Reduce  Total  Ownership  Cost 

•  Manage  aviation  maintenance  practices  to  maximize  aircraft  availability 

•  Identify  and  reduce  non  value  added  process  steps  (waste) 

•  Manage  inventory  and  investments  (parts,  equipment,  and  facilities) 

•  Manage  and  reduce  the  variability  in  processes 

•  Create  a  culture  of  CPI 

Each  MALS  has  a  dedicated  AIRSpeed  office.  This  office  is  designed  to  be  led  by 
an  AIRSpeed  officer  with  a  minimum  of  three  other  personnel.  This  team  will  consist  of 
at  least  two  Green  Belt-certified  personnel  and  one  Black  Belt-trained  individual. 
Furthermore,  each  division  within  the  MALS  is  to  have  two  Green  Belt  certified 
personnel. 

The  AIRSpeed  office  offers  Yellow  Belt  and  Green  Belt  Lean  Six  Sigma  training. 
The  trainees  are  then  encouraged  to  return  to  their  divisions  to  implement  process 
improvement  events.  Identification  of  projects  can  happen  in  various  ways.  Most  often, 
Marines  recognize  areas  for  improvement  in  their  own  divisions  simply  by  being  subject 
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matter  experts  and  knowing  what  is  and  is  not  working.  They  experience  the  constraints 
on  a  day-to-day  basis.  The  AMO  has  a  high-level  view  of  the  maintenance  department 
and  is  able  to  identify  problems  that  affect  multiple  divisions  or  departments.  The 
enterprise  project  alignment  tool  (EPAT)  is  another  way  that  projects  are  identified. 

The  EPAT  is  used  by  an  AIRSpeed  office  to  focus  its  efforts.  It  “translates 
strategic  level  objectives  to  tactical  goals  and  tasks”  (NAVAIR  6.7.2. 1,  2013,  p.  G-12). 
Within  the  EPAT,  there  are  three  primary  documents:  goal  alignment  chart  (GAC),  site 
opportunity  worksheet  (SOW),  and  tracking  matrix  (TM).  The  example  EPAT  in  Figure 
10  demonstrates  the  interaction  of  various  organizations’  goal  and  objectives. 

The  GAC  is  a  command-level  document  that  identifies  process  improvement 
opportunities  that  are  aligned  with  strategic  goals.  The  SOW  is  the  AIRSpeed  officer’s 
tool  for  tracking  projects  that  will  help  meet  the  strategic  goals.  The  TM  is  the  tool  that 
portrays  project  progress  as  well  as  how  the  AIRSpeed  projects  are  aligned  with  NAE 
objectives,  command  goals,  and  department  goals. 


Dept. 

Goals 


Site  Opportunity 
Worksheet 

AIRSpeed 

Projects 

1.  RIE  Event  m  62  \ 

2.  BB  Project  VLK  9 

3.  RIE  in  Medical 

4.  VSA  in  440 

5.  RIE  in  Eng. 

_ —V 


NAE 

Objectives 


Command 

Goals 


Enable  Navy  &  Marine  Corps  to  meet 
core  capabilities 


Figure  10.  Example  Project  Alignment  Tool 
Source:  NAVAIR  6.7.2. 1  (2013). 
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I. 


CURRENT  READINESS 


The  Marine  Corps’  Current  Readiness  program  is  a  division  of  the  Naval  Aviation 
Enterprise  (NAE)  tasked  to  “to  improve  the  delivery  of  combat  ready  forces  to  meet 
current  and  future  operational  requirements  at  an  optimal  Operating  and  Support  (O&S) 
cost”  (Naval  Aviation  Enterprise,  n.d.).  The  Current  Readiness  program  spans  the 
entire  system  of  Naval  Aviation.  The  metrics  used  in  this  program  inform  Naval  Aviation 
leaders  of  the  current  standing  of  a  T/M/S  fleet  in  terms  of  readiness,  performance, 
and  competency. 

Some  of  the  key  metrics  used  are  Aircrew  Core  Competency,  Maintainer  Core 
Competency,  Aircraft  Availability,  and  Cost  per  Flight  Hour.  These  metrics  are  separated 
per  T/M/S  and  examined  at  a  “per  squadron”  level  as  well  as  a  “total  fleet”  level.  All  of 
these  metrics  are  in  relation  to  a  respective  achievement  target. 

The  Current  Readiness  program  also  examines  cost  and  schedule  performance  of 
the  T/M/S  fleet,  using  popular  Earned  Value  Management  principles.  The  cost 
performance  index  is  calculated  based  on  planned  versus  actual  total  cost  of  contracts, 
fuel,  repairables,  and  consumables.  The  schedule  performance  index  is  calculated  based 
on  the  flight  hours  programmed  versus  actually  flown  for  a  given  fiscal  year.  The  actual 
cost  in  relation  to  the  actual  executed  flight  hours  provides  the  Current  Readiness  with  an 
Execution  Index. 

If  the  Execution  Index  is  above  or  below  a  certain  threshold,  leaders  dig  deeper 
into  the  root  cause  of  the  variation.  A  cycle  of  briefs  occurs  throughout  the  year  that 
spans  the  entire  chain  of  command  in  Naval  Aviation.  These  briefs  are  a  collaborative 
effort  of  all  levels  in  Navy  and  Marine  aviation  to  address  the  root  cause  of  problems  that 
are  negatively  affecting  readiness  or  costs  in  the  fleet. 

J.  DOD  SUPPLY  CHAIN  IMPLEMENTATION  GUIDE 

In  2000,  the  Logistics  Management  Institute  (LMI)  developed  the  Department  of 

Defense  (DOD)  Supply  Chain  Management  Implementation  Guide.  This  report 

acknowledges  “the  lack  of  correct  and  comprehensive  supply  chain  metrics  in  DOD” 

(Logistics  Management  Institute,  2000).  It  goes  on  to  suggest  that  many  of  the  metrics 
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used  in  supply  chain  management  are  financial  or  accounting  measures  inappropriately 
applied  to  supply  chain  manamagement.  This  is  evidenced  by  “a  consensus  in  DOD  [that] 
considers  the  metrics  currently  available  to  senior  DOD  managers  to  be  inadequate  or 
lacking  the  depth  to  measure  effectivenes  and  efficiency  across  the  DOD  supply  chain” 
(Logistics  Management  Institute,  2000,  p.  30).  LMI  suggests,  therefore,  that  supply 
chains  be  measured  on  the  basis  of  ten  service  quality  factors:  tangibles,  reliability, 
responsiveness,  competence,  courtesy,  credibility,  security,  access,  communication,  and 
understanding  the  customer. 

Balestreri  and  McDoniel  apply  LMI’s  strategy  to  supply  chain  management  to  a 
MALS  by  adapting  the  Supply-Chain  Operations  Reference  model  (SCOR),  a 
performance  measurement  system  popular  in  civilian  industry.  They  defined  the  overall 
goal  of  AVLOG  to  be  aircraft  readiness  (Balestreri  &  McDoniel,  2002).  A  MALS  is  only 
one  factor  of  many  that  can  enable  a  flight  line  squadron  to  increase  aircraft  readiness, 
however.  Therefore,  they  conclude  that  inventory  management  and  production 
management  are  the  “primary  critical  success  factors”  for  contributing  to  aircraft 
readiness  and  “logistics  responsiveness  and  material  availability  [are]  key  conditions  for 
achieving  success”  (Balestreri  &  McDoniel,  2002,  p.83). 

Logistics  Responsiveness  was  simply  expeditiously  delivering  parts  to  the 
customer  when  requested,  and  material  availability  was  comprised  of  1) 
ensuring  at  least  one  part  was  available  for  issue  off  the  shelf  and  2) 
reducing  DIFM  material  (due-in-from-maintenance  84  or  work  in  process 
inventory)...  These  concepts  follow  the  adapted  SCOR  logistic 
performance  metrics  of  Supply  Chain  Response  Time  and  Perfect  Order 
Fulfillment.  (Balestreri  &  McDoniel,  2002,  p.  84) 

They  state  that  logistics  response  time  (LRT)  and  customer  wait  time  (CWT)  are 
“the  most  significant  metrics  the  MALS  can  utilize  to  improve  aircraft  Ao  with  respect  to 
responsiveness”  (p.  87).  Balestreri  and  McDoniel  define  LRT  for  the  MALS  as  “the  total 
average  time  spent  on-station  from  initiation  of  a  squadron’s  requisition  to  the  delivery 
and  completion  of  that  requisition,”  and  they  define  CWT  as  “the  total  average  [time] 
spent  processing  customer’s  requisition  on-station  and  ultimately  satisfying  that 
requisition  from  off-station  or  upstream  members  of  the  Naval/Marine  Corps  aviation 

logistics  supply  chain”  (p.  86).  Succinctly,  LRT  is  the  amount  of  time  a  requisition 

17 


spends  on-station  and  CWT  is  the  time  spent  off-station.  The  combination  of  these  times 
is  the  Supply  Chain  Response  Time. 

An  additional  metric  they  emphasize  is  perfect  order  fulfillment,  defined  as  “the 
ratio  of  perfectly  satisfied  orders  to  all  orders  measured”  (p.  93).  This  is  generally 
equivalent  to  a  common  fill  rate  metric  used  in  industry  sectors.  The  metric  reflects 
material  availability  as  a  success  factor  in  contributing  to  aircraft  readinesss. 

Perfect  order  fulfillment  and  supply  chain  response  time  are  two  metrics  that, 
when  optimized,  will  contribute  to  increased  aircraft  readiness,  the  goal  of  AVLOG. 
Balestreri  and  McDoniel  conclude  that  “perfect  order  fulfillment  and  supply  chain 
response  time  must  be  integrated  throughout  not  only  the  MALS,  but  also  throughout  the 
entire  Naval/Marine  Corps  aviation  logistics  supply  chain.  Utilizing  these  two 
performance  metrics  throughout  the  entire  enterprise  will  focus  all  members  of  the  supply 
chain  on  the  common  goal  of  aircraft  readiness”  (p.  112). 

K.  CHAPTER  SUMMARY 

The  nodal  logistics  laydown  structure  of  MALSP  MOD  requires  that  all  members 
in  the  logistics  chain  are  working  toward  the  same  goal  in  order  to  be  successful.  To 
measure  the  progress  toward  that  goal,  data  management  systems  such  as  BMT, 
NALCOMIS,  and  RSUPPLY  have  been  established  to  document  and  measure  the  major 
functions  of  the  MALS.  Programs  such  as  AIRSpeed  and  Current  Readiness  have  been 
established  to  improve  processes  and  appropriately  allocate  resources  from  a  systematic 
level.  These  systems  and  programs  are  established  to  align  all  of  AVLOG  to  the  common 
goal  of  sustaining  aircraft  readiness  at  a  high  enough  level  to  enable  aircrew  to  achieve 
and  maintain  appropriate  competencies. 
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III.  LITERATURE  REVIEW 


A.  INTRODUCTION 

This  chapter  discusses  current  measurement  systems  that  are  considered  best 
business  practices  in  the  industry  realm.  It  also  discusses  two  different  approaches  to 
performance  measurement  adopted  by  the  Air  Force  and  the  Navy  submarine  fleet.  An 
understanding  of  these  performance  measurement  systems  will  aid  in  the  discussion  of 
what  may  and  may  not  be  applicable  to  a  MALS. 

A  popular  adage  among  military  and  business  professionals  states  what  you 
measure  is  what  you  will  get.  “Inspect  what  you  expect”  is  a  version  of  this  concept  often 
repeated  in  the  Marine  Corps.  Eliyahu  Goldratt,  who  popularized  the  theory  of  constraints 
management  principle,  wrote,  “Tell  me  how  you  measure  me,  and  I  will  tell  you  how  I 
will  behave”  (Goldratt,  1990,  p.  26).  The  link  between  measurements  and  performance 
has  led  to  the  creation  of  many  measurement  systems  in  both  civilian  and  military 
organizations.  This  literature  review  examines  logistics  and  supply  chain  performance 
measurements  and  their  use  among  civilian  and  military  organizations. 

B.  METRICS  AND  MEASUREMENT  SYSTEMS 

There  is  a  necessary  distinction  to  be  made  between  performance  metrics  and 
performance  measurement  systems.  Managers  may  easily  make  the  mistake  of  assigning 
a  single  performance  metric  as  their  performance  measurement  system,  using  an 
individual  part  to  be  indicative  of  the  whole.  A  performance  metric,  however,  is  a  metric 
used  within  a  performance  measurement  system  and  largely  focuses  on  measuring 
individual  processes  (Caplice  &  Sheffi,  1994). 

Logistics  does  not  consist  of  a  singular  activity,  however,  and  thus  a  singular 
measurement  is  insufficient  to  measure  a  system’s  performance.  Logistics  “encompasses 
a  complex  set  of  activities  which  require  a  collection  of  metrics  to  adequately  measure 
performance”  (Caplice  &  Sheffi,  1995,  p.  61).  A  performance  measurement  system  is 
built  upon  performance  metrics. 
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Not  all  performance  metrics  and  performance  measurement  systems  are  helpful, 
however.  Caplice  and  Sheffi  provide  separate  criteria  for  evaluating  performance  metrics 
and  performance  measurement  systems.  The  eight  criteria  for  evaluating  individual 
logistic  performance  metrics  are  defined  in  Table  2. 


Table  2.  Eight  Performance  Metric  Evaluation  Criteria. 

Adapted  from  Caplice  and  Sheffi  (1994). 


Performance  Metric  Evaluation  Criteria  Summary 

Criteria 

Description 

Validity 

The  metric  accurately  captures  the  events  and  activities  being 
measured  and  controls  for  any  exogenous  factors. 

Robustness 

The  metric  is  interpreted  similarly  by  the  users,  is  comparable  across 
time,  location,  &  organizations,  and  is  repeatable. 

Usefulness 

The  metric  is  readily  understandable  by  the  decision  maker  and 
provides  a  guide  for  action  to  be  taken. 

Integration 

The  metric  includes  all  relevant  aspects  of  the  process  and  promotes 
coordination  across  functions  and  divisions. 

Economy 

The  benefits  of  using  the  metric  outweigh  the  costs  of  data 
collection,  analysis,  and  reporting. 

Compatibility 

The  metric  is  compatible  with  the  existing  information,  material,  and 
cash  flows  and  systems  in  the  organization. 

Level  of 
Detail 

The  metric  provides  a  sufficient  degree  of  granularity  or  aggregation 
for  the  user. 

Behavioral 

Soundness 

The  metric  minimizes  incentives  for  counter-productive  acts  or 
game -playing  and  is  presented  in  useful  form. 

Not  all  performance  measurement  systems  are  equally  helpful,  either.  The  system 
can  be  “well  designed  at  the  strategic  level  [but]  can  be  flawed  at  the  individual  metric 
level”  (Caplice  &  Sheffi,  1994,  p.  11).  Six  evaluation  criteria  for  performance 
measurement  systems  are  shown  in  Table  3.  Crafting  performance  measurement  systems 
according  to  these  criteria  will  help  ensure  the  resulting  actions  of  those  measured  by  the 
system  will  best  benefit  the  organization  and  its  customers. 
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Table  3.  Six  Performance  Measurement  System  Criteria. 

Adapted  from  Caplice  and  Sheffi  (1995). 


Performance  Measurement  System  Evaluation  Criteria  Summary 

Criteria 

Description 

Comprehensive 

The  measurement  system  captures  all  relevant  constituencies  and 
stakeholders  for  the  process. 

Causally  Oriented 

The  measurement  system  tracks  those  activities  and  indicators 
that  influence  future,  as  well  as  current,  performance. 

Vertically 

Integrated 

The  measurement  system  translates  the  overall  firm  strategy  to  all 
decision  makers  within  the  organization  and  is  connected  to  the 
proper  reward  system. 

Horizontally 

Integrated 

The  measurement  system  includes  all  pertinent  activities, 
functions,  and  departments  along  the  process. 

Internally 

Comparable 

The  measurement  system  recognizes  and  allows  for  tradeoffs 
between  the  different  dimensions  of  performance. 

Useful 

The  measurement  system  is  readily  understandable  by  the 
decision  makers  and  provides  a  guide  for  action  to  be  taken. 

No  organization  can  excel  completely  in  all  criteria,  whether  for  performance 
metrics  or  performance  measurement  systems.  Necessary  tradeoffs  must  be  made  among 
the  metric  criteria.  The  integration  and  usefulness  criteria,  for  example,  face  a  necessary 
trade-off  between  scope  and  span  of  control  (Caplice  &  Sheffi,  1994).  Additionally,  as  a 
metric  becomes  more  robust,  it  allows  for  more  comparability  between  organizations 
or  units.  However,  this  negatively  impacts  its  validity  as  it  has  to  be  less  specific  to 
an  organization. 

The  goal  of  a  metric  is  to  be  indicative  of  the  level  of  performance  of  an 
individual  process.  The  goal  of  a  measurement  system,  however,  is  to  “guide  and 
influence  the  decision  making  process”  (Caplice  &  Sheffi,  1995,  p.  65).  In  order  properly 
affect  decision  makers,  a  performance  measurement  system  ought  to  be  developed  in 
terms  of  output  and  the  customer. 

Regarding  output,  “the  basic  transaction  [for  logistics]  is  a  completed  delivery  to 


a  customer”  (1995,  p.  65).  The  output  (i.e.,  completed  deliveries)  and  the  customer  are  to 

be  the  focal  points  of  a  performance  measurement  system  because  the  output  is  a  result  of 
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all  processes  within  an  organization  and  the  delivery  to  a  customer  is  the  extent  to  which 
a  logistics  system  has  influence.  A  diagram  illustrating  how  a  system’s  output  is  a 
function  of  individual  processes  is  seen  in  Figure  11. 


Function  Versus  Process  Output  For  Logistics 


Outbound  Logistics  Process 
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Figure  11.  Process  Output  as  a  Product  of  Function  Outputs. 

Source:  Caplice  and  Sheffi  (1995). 


C.  COMMON  PERFORMANCE  MEASUREMENT  SYSTEMS 

Popular  measures  of  performance  in  the  civilian  industry  realm  include  the 
Supply  Chain  Measurement  System  (SCM),  Balanced  Scorecard,  Economic  Value  Added 
(EVA),  Process-Based  Measurement,  and  Supply-Chain  Operations  Reference  model 
(SCOR).  Companies  adopt  and  adapt  these  systems  to  create  a  customized  fit  for  their 
organization  that  meets  their  specific  needs. 

Piotrowicz  and  Cuthbertson  (2014)  surveyed  51  international  companies 
regarding  the  metrics  they  use,  the  factors  that  influence  those  metrics,  and  their 
perceived  importance  to  the  organization.  These  companies  described  their  core  activities 
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as  manufacturing,  retail/wholesaling,  transport,  logistics  value  added,  warehousing,  or 
some  combination  thereof.  Of  these  companies,  the  performance  measurement  systems 
used  most  often  were  the  Balanced  Scorecard  (19  companies),  Process-Based 
Measurement  (15  companies),  and  SCOR  model  (12  companies). 

The  Balanced  Scorecard  was  developed  by  Kaplan  and  Norton  (1992)  to  be  “a  set 
of  measures  that  gives  top  managers  a  fast  but  comprehensive  view  of  the  business”  (p. 
71).  It  provides  insight  into  the  performance  of  an  organization  through  four  difference 
perspectives:  customer  perspective,  internal  perspective,  innovation  and  learning 
perspective,  and  financial  perspective.  As  seen  in  Figure  12,  Balanced  Scorecard  seeks  to 
focus  an  organization  on  overarching  goals  and  strategy  while  traditional  measurement 
systems  focus  on  individual  metrics.  Traditional  measurement  systems  may  cause 
employees  to  work  to  optimize  the  metric  rather  than  the  goal  it  is  meant  to  serve.  The 
Balanced  Scorecard  focuses  the  whole  organization  on  a  unifying  goal. 


Financial  Perspective 
Goals  and  Measures 

✓  \ 


Customer 

Perspective 

Internal  Business 
Perspective 

Goals  and  Measures 

Goals  and  Measures 

\  / 

Innovation  and 
Learning  Perspective 

Goals  and  Measures 


Figure  12.  Balanced  Scorecard  Diagram.  Adapted  from  Balanced  Scorecard  Institute 

(2015). 
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The  Balanced  Scorecard  is  helpful  because  the  organization  must  define  goals  not 
only  in  terms  of  financial  performance  (which  is  easily  measured  and  understood  by 
stakeholders)  but  in  other  more  intangible  aspects  of  a  corporation’s  performance.  By 
setting  goals  in  all  four  perspectives,  a  company  must  also  determine  measures  that 
convey  effectiveness  in  achieving  those  goals. 

The  third  primary  measurement  system  respondents  used  was  the  Process-Based 
Measurement  system.  It  takes  an  analytical  approach  that  focuses  on  the  supply  chain  as  a 
process  divided  into  six  main  sections:  supplier,  inbound  logistics,  core  manufacturer, 
outbound  logistics,  marketing  and  sales,  and  end  customers  (Chan  &  Qi,  2003).  Processes 
are  decomposed  into  core  processes,  sub-processes,  and  activities,  driven  by  goals  and 
defined  in  terms  of  responsibility  and  function,  as  shown  in  Figure  13.  A  primary 
advantage  of  a  process-based  measurement  system  is  that  it  creates  opportunity  for 
continuous  process  improvement.  The  Balanced  Scorecard  focuses  on  prior  performance 
and  is  not  able  to  readily  identify  problems  in  the  supply  chain  process.  Process-based 
measurement  utilizes  a  metric  called  Performance  of  Activity  (POA).  These 
measurements  include  tangible  and  intangible  aspects  of  the  process  and  include  cost, 
time,  capacity,  capability,  productivity,  utilization,  and  outcome. 


SCM  Context 


Figure  13.  Process-Based  Measurement  System. 

Source:  Chan  and  Qi  (2003). 
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The  SCOR  model  is  also  a  process  focused  measurement  system  but  it  focuses  on 
four  main  processes:  plan,  source,  make,  and  deliver.  The  model  continues  to  break  those 
four  main  categories  into  two  more  levels  of  sub-processes  but  those  four  functions  are 
the  primary  focus,  as  seen  in  Figure  14.  SCOR  “spans  interactions  pertaining  to 
customers/markets  and  transactions  pertaining  to  products”  (Khare,  Saxsena,  &  Teeware, 
2012,  p.  29).  While  the  SCOR  model  rated  third  in  performance  measurement  systems  by 
companies  in  the  Piotrowicz  study,  specific  SCOR  metrics  were  used  the  most 
individually. 


Plan 


Figure  14.  The  SCOR®  Model.  Source:  Defense 
Acquisition  University  (2012) 

D.  SCHEDULING  AND  DISPATCH  RULES 

Maintenance  managers  are  continually  faced  with  decisions  about  workload 
scheduling  and  which  items  should  take  priority  for  repair.  Various  methods  exist,  called 
dispatch  rules,  that  aide  decision  makers  in  this  task.  Examples  of  popular  dispatch  rules 
include  prioritizing  jobs  by  shortest  processing  time,  longest  processing  time,  earliest  due 
date,  first-come-first-serve,  and  critical  ratio  (Baker  &  Trietsch,  2009,  p.  359). 

Dispatch  rules  are  helpful  for  managers  because  they  enable  quick  decision 
making  for  prioritizing  jobs.  Each  time  a  job  is  completed,  a  new  priority  must  be 
determined.  If  the  dispatch  rule  of  shortest  processing  time  is  followed,  then  the 
estimated  processing  times  of  all  jobs  are  examined  and  priority  goes  to  the  job  with  the 
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shortest  time.  The  inverse  is  true  of  the  longest  processing  time.  The  earliest  due  date 
dispatch  rule  prioritizes  jobs  based  upon  their  due  date  and  disregards  their  estimated 
processing  time  (2009).  A  first-come-first-serve  dispatch  rule  prioritizes  jobs  in  the  order 
in  which  they  are  received. 

The  critical  ratio  dispatch  rule  is  a  ratio  of  remaining  allowance  and  remaining 
work.  Baker  and  Trietsch  state,  “critical  ratio  priorities  measure  urgency  by  the  ratio  of 
remaining  allowance  and  remaining  work  rather  than  their  difference”  (p.  359).  Those 
with  the  smallest  critical  ratio  receive  priority  while  a  ratio  value  of  1  indicates  a  job  has 
the  most  work  left  to  do  but  also  the  most  time  in  which  to  do  it. 

E.  SERVICE  LEVEL 

Service  levels  and  fill  rates  are  common  metrics  used  in  supply  chain 
management  to  measure  the  effectiveness  of  an  inventory  management  policy.  Nahmias 
(2009)  titles  these  metrics  as  Type  1  Service  (also  known  as  Service  Level)  and  Type  2 
Service  (also  known  as  Fill  Rate).  Service  level  is  “the  probability  of  not  stocking  out  in 
the  lead  time”  (p.  272).  Fill  rate  is  defined  as  “the  proportion  of  demands  that  are  met 
from  stock”  (p.  273). 

The  fill  rate  is  a  commonly  used  service  metric  and  is  generally  what  is  meant 
when  a  manager  speaks  of  service  level  (Nahmias,  2009).  This  metric  is  important  for 
determining  inventory  policy  in  order  to  ensure  a  certain  level  of  customer  satisfaction  is 
met.  The  cost  to  provide  100%  satisfaction  can  often  be  too  great  for  an  organization  to 
absorb  so  a  certain  level  of  risk  must  be  accepted  on  behalf  of  an  organization.  The  risk 
tolerance  of  an  organization  is  generally  determined  by  the  cost  of  that  risk. 

For  example,  if  a  shortage  cost  is  extremely  high  and  a  holding  cost  is  extremely 
low,  then  it  is  easy  for  an  organization  to  justify  a  high  level  of  inventory  in  order  to 
optimize  customer  satisfaction.  If  shortage  costs  are  low  and  holding  costs  are  high,  it  is 
easier  to  justify  accepting  more  risk  for  the  sake  of  optimizing  financial  performance. 
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F.  LEAN  SIX  SIGMA  AND  THEORY  OF  CONSTRAINTS 

Lean  Six  Sigma  and  the  Theory  of  Constraints  are  not  performance  measurement 
systems  but  they  are  widely  used  within  logistics  and  supply  chain  management.  Their 
relationship  with  performance  measurement  is  quite  direct,  however.  The  Theory  of 
Constraints  focuses  on  managing  the  constraints  in  a  process  in  order  to  improve 
productivity.  Lean  methodology  seeks  to  eliminate  waste  within  a  process  and  therefore 
reduce  cycle  time.  Six  Sigma  is  a  statistical  control  process  that  focuses  on  reducing 
variation  in  a  process  that  can  be  costly. 

To  implement  any  of  these  process  improvement  methodologies,  one  must  define 
the  goal  of  the  process  as  well  as  determine  how  to  measure  that  goal.  Once  that  goal  is 
defined,  an  organization  can  begin  improving  processes  that  specifically  contribute  to 
achieving  that  goal.  The  DMAIC  model  of  process  improvement  (Define,  Measure, 
Analyze,  Improve,  and  Control)  uses  these  methodologies  to  optimize  a  process 
according  to  a  specific  goal.  One  must  define  tolerances  in  a  process  in  order  to 
implement  Six  Sigma  and  one  must  define  an  overall  measure  of  performance  according 
to  a  goal  in  order  to  implement  Theory  of  Constraints. 

G.  AIR  FORCE 

The  Air  Force  Logistics  Management  Agency  (AFLMA)  and  the  Air  Force 
Materiel  Command  (AFMC)  both  developed  and  recommended  separate  performance 
measurement  systems  for  use  by  the  Air  Force.  Leonard  (2004)  examined  these  two 
performance  measurement  systems  and  found  that  the  AFMC  system  is  modeled  around 
four  primary  metrics:  Aircraft  Availability,  Requirements  Computation,  Asset  Allocation 
and  Funding,  and  Real  World  Performance.  The  AFLMA  focused  rather  on  four  core 
processes:  Repair,  Buy,  Stockage  and  Distribution,  and  Funding. 

AFLMA’ s  performance  measurement  system  espouses  a  Balanced  Scorecard 
approach  with  six  segments,  as  seen  in  Figure  15.  These  six  segments  have  23  assigned 
metrics  in  total  measuring  each  segment’s  performance  (Leonard,  2004). 
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Figure  15.  AFLMA’s  Balanced  Scorecard. 

Source:  Leonard  (2004). 

The  AFMC  performance  measurement  system  uses  only  ten  metrics,  five 
measuring  performance  and  five  measuring  process,  as  depicted  in  Figure  16.  The 
simplicity  of  this  model  may  assist  managers  in  the  decision  making  process  more  so  than 
AFLMA’s  measurement  system  of  23  metrics. 
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Figure  16.  Collective  Overview  of  AFMC’s  Measurement  System. 

Source:  Leonard  (2004). 

H.  NAVY  SUBMARINE  FLEET  MAINTENANCE  METRICS 

The  Navy  created  a  Submarine  Fleet  Maintenance  Metric  Working  Group 
(Working  Group)  tasked  with  developing  various  metrics  to  measure  the  performance  of 
submarine  Fleet  Maintenance.  The  Working  Group  “took  the  perspective  of  viewing 
Fleet  Maintenance  as  a  complex  system,  with  multiple  stakeholders,  executing  a  series  of 
integrated  processes”  (Submarine  Fleet  Maintenance  Metric  Working  Group,  2013,  p.  ii). 

The  systems  approach  enabled  the  Working  Group  to  create  an  “integrated  set  of 
metrics  that  measure  performance,  productivity,  planning,  execution,  finance,  and 
backlog,  while  reducing  the  impact  on  the  already  limited  bandwidth  of  those  overseeing 
intermediate  level  maintenance”  (Submarine  Fleet  Maintenance  Metric  Working  Group, 
2013,  p.  ii).  This  systemic  view  and  the  integration  of  metrics  enables  results  in 
performance  and  productivity  metrics  that  are  positively  and  negatively  affected  by  the 
more  discrete  metrics  of  which  they  consist. 
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The  Working  Group  developed  two  primary  metrics,  a  productivity  metric  and  a 
performance  metric.  They  generally  define  the  productivity  metric  (PROD)  as  “output 
divided  by  input.”  It  consists  of  four  variables,  demonstrated  in  Equation  1:  routine 
maintenance  (2K),  periodic  maintenance  requirements  (PMR),  and  alterations  (ALTS), 
divided  by  the  total  direct  cost  (CTC)  (Submarine  Fleet  Maintenance  Metric  Working 
Group,  2013). 

+  PMR  +  ALTS) 

Productivity  —  - - 

Equation  1.  Working  Group  Productivity  Metric 
Source:  Submarine  Fleet  Maintenance  Metric  Working  Group  (2013) 

The  second  primary  metric  the  Working  Group  developed  was  the  performance 
metric  (PERF).  It  is  defined  as  “the  weighted  sum  of  on-time  delivery  (TD),  total  direct 
cost  (COST),  planning  event  completion  (PLNG),  late  work  (LW),  deferred  work  (DW), 
and  backlog  (BL)  divided  by  the  sum  of  the  weights”  (Submarine  Fleet  Maintenance 
Metric  Working  Group,  2013,  p.  38).  Mathematically,  it  is  defined  as: 

.42077)  +  222C0ST  +  .132  PLNG  +  .082 LW  +  .072DW  +  .072 BL 
Performance  =  - - 

Equation  2.  Working  Group  Performance  Metric 
Source:  Submarine  Fleet  Maintenance  Metric  Working  Group  (2013) 

The  variables  in  PERF  go  beyond  typical  schedule  and  cost  variables.  Other 
variables  were  added  that  were  considered  highly  important  by  the  Working  Group.  The 
basic  PERF  metric  was  given  several  different  varieties  to  better  serve  the  concerns  of 
different  stakeholders  in  the  system.  For  example,  activities  within  the  logistic  system  are 
affected  differently  by  late  work  and  therefore  the  PERF  metric  weights  that  variable 
differently  to  better  capture  the  magnitudes  by  which  late  work  affects  their  performance 
individually.  The  basic  PERF  metric  format,  however,  is  largely  the  same  for  each 
activity.  The  weights  for  each  variable  are  assigned  according  to  the  opinions  of  the 
subject  matter  experts  in  the  Working  Group. 
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I.  CHAPTER  SUMMARY 

Logistics  metrics  and  performance  measurement  systems  are  not  equivalent.  For 
either  of  them  to  be  useful  to  decision  makers,  they  ought  to  sufficiently  meet  the  criteria 
proposed  by  Caplice  and  Sheffi  (1994).  In  recognition  of  the  delineation  between 
individual  metrics  and  measurement  systems,  civilian  industry  has  adopted  several 
performance  measurement  systems.  These  systems  provide  holistic  solutions  to  measure 
the  key  components  of  a  business  that  contribute  most  directly  to  an  organization’s  goal. 

These  systems,  however,  largely  consist  of  certain  financial  metrics  that  are  not 
directly  applicable  to  many  military  organizations.  Given  these  shortcomings,  the  Navy 
submarine  maintenance  fleet,  for  example,  created  a  performance  metric  that  met  their 
specific  needs.  This  performance  metric  assigns  weights  to  key  logistic  metrics  and 
reports  the  weighted  average  as  a  holistic  measurement  of  performance. 
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IV.  ANALYSIS 


A.  INTRODUCTION 

How  one  is  measured  influences  how  one  behaves.  This  underlying  idea  is  the 
foundation  of  this  section.  An  organization  must  choose  carefully  which  measurements  it 
uses  to  determine  how  it  is  performing.  These  measurements,  when  emphasized,  will 
influence  an  individual’s  behavior,  the  performance  of  an  organization,  and  the  strategy 
an  organization  adopts.  This  analysis  section  examines  primary  metrics  currently  used  by 
a  MALS  and  how  they  affect  individuals,  the  organization,  and  the  customer. 

B.  DEFINING  THE  GOAL  OF  A  MALS 

Balestreri  and  McDoniel  (2002)  may  very  well  be  correct  in  concluding  that 
aircraft  readiness  is  the  goal  of  aviation  logistics  as  a  whole.  However,  defining  this  as 
the  goal  of  a  MALS,  specifically,  is  problematic.  There  is  an  important  disconnect 
between  a  MALS  and  aircraft  readiness  that  must  be  recognized. 

The  performance  of  a  MALS  may  directly  contribute  to  aircraft  readiness  but  it 
cannot  solely  cause  aircraft  readiness,  though  a  well-perforing  MALS  is  necessary  for 
high  aircraft  readiness.  If  a  MALS  can  solely  cause  aircraft  readiness  then  an  optimized, 
perfectly  performing  MALS  in  ideal  conditions  would  result  in  perfect  aircraft  readiness 
among  the  flightline  squadrons  it  supports.  This,  however,  is  not  the  case.  An  optimized 
MALS  will  result  in  RFI  parts  being  delivered  to  flight  line  squadrons  as  soon  as  they  are 
needed  by  those  squadrons,  enabling  though  squadrons  to  achieve  aircraft  readiness.  That 
is  the  extent  of  the  output  of  a  MALS. 

MALS  certainly  performs  more  functions  for  flight-line  squadrons  than  delivering 
RFI  parts.  MALS  Marines  may  conduct  nondestructive  inspections  (NDI)  of  helicopter 
blades  or  other  airframe  components.  They  may  help  troubleshoot  low  power  on  an 
engine.  There  are  many  functions  a  MALS  serves  for  their  flight  line  squadrons  but  the 
vast  majority  of  them,  in  some  sense,  result  in  an  RFI  part  being  delivered  for  a  part 
being  deemed  RFI.  Determining  a  blade  is  safe  for  flight  through  an  NDI  is  a  form  of 
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rendering  the  potentially  non-RFI  blade  as,  in  fact,  RFI.  Helping  troubleshoot  low  power 
engines  and  bringing  them  up  to  normal  power  is  a  form  of  making  that  engine  RFI. 

The  MALS’s  extent  of  influence  on  readiness  in  a  flight  line  squadron  ends  once 
the  ready-for-issue  (RFI)  part  is  delivered  to  the  flight  line  squadron.  The  squadron  must 
then  install  the  part,  the  correctly  installed  part  must  actually  solve  the  discrepancy  for 
which  the  part  was  ordered,  and,  depending  on  the  level  of  maintenance  performed,  the 
mainteners  and  aircrew  may  need  to  successfully  perform  a  functional  check  flight.  Only 
then  will  that  aircraft  be  able  to  contribute  to  an  increase  aircraft  readiness.  The  goal  of 
aviation  logistics  as  a  whole  may  be  to  increase  aircraft  readiness  but  that  is  beyond  the 
scope  of  a  MALS. 

While  a  MALS  has  many  functions  and  responsibilities  (i.e.,  maintenance  on 
repairable  components,  inventory  management,  supply  chain  management,  supporting 
squadron  detachments  and  deployments,  training  Marines,  ensuring  the  welfare  of 
Marines  and  their  families),  the  ultimate  goal  of  a  MALS  is  to  provide  ready-for-issue 
(RFI)  parts  to  flight  line  squadrons.  An  organization  has  many  responsibilities  that  do  not 
contribute  to  achieving  its  goal  but  those  responsibilities  must  not  be  misunderstood  as 
that  organization’s  goal.  The  Marine  Corps,  for  example,  has  not  established  a  MALS  in 
Hawaii  in  order  to  train  maintenance  Marines.  That  is  merely  one  of  its  responsibilities.  A 
MALS  exists  in  Hawaii  to  best  support  the  flight  line  squadrons  there  with  the  parts  that 
they  need. 

The  goal  of  the  MALS  must  not  be  confused  with  its  various  responsibilties. 
Further,  the  goal  of  the  MALS  cannot  exceed  the  end  point  of  the  supply  chain  process 
that  it  is  able  to  control.  That  end  point  is  when  an  RFI  part  is  delivered  to  the  squadron 
or  an  I-Level  maintainer  is  able  to  sign  off  a  MAF  submitted  to  the  MALS  from  a 
squadron.  The  goal  of  the  MALS,  therefore,  is  the  output  of  its  functions — specifically, 
RFI  parts. 

C.  DEFINING  A  GOOD  METRIC 

Currently,  a  MALS  has  metrics  measuring  its  many  individual  functions  or 
processes.  Identifying  an  overarching  performance  measurement  system  is  much  more 
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difficult.  For  example,  the  Buffer  Management  Tool,  discussed  in  this  chapter,  best 
manages  pertinent  information  and  measurements  of  primary  functions  in  a  MALS,  yet  it 
is  not  necessarily  used  as  a  performance  measurement  system.  Rather,  individual  data 
process  measurements  from  the  BMT  are  most  commonly  captured  and  analyzed. 

MALS  performance  as  a  whole  ought  not  to  be  measured  solely  in  terms  of  TRR, 
BCM  rates,  or  any  other  individual  metric.  These  metrics  may  be  important  for  internal 
analysis  but  they  do  not  reflect  the  level  of  success  a  MALS  has  in  obtaining  its  goal.  For 
example,  black  MAFs  in  regard  to  Design  TRR  are  a  maintenance  specific  metric 
whereas  Supply  Effectiveness  is  a  supply  focused  metric  (it  is  affected  by  maintenance 
performance  but  that  is  not  the  focus  of  the  metric).  This  does  not  mean,  however,  that 
these  metrics  are  to  be  disregarded.  It  simply  means  these  metrics  are  indicative  of 
internal  process  performance  only. 

Caplice  and  Sheffi  (1994)  do  not  necessarily  distinguish  between  criteria  that 
make  good  logistics  metrics  and  criteria  that  make  good  performance  measurement 
systems.  The  distinction  is  necessary  because  the  purposes  of  each  are  different.  A 
logistics  metric  is  internally  focused,  examining  the  efficiency  or  productivity  of  a 
process.  A  performance  measurement  system  examines  the  overall  system’s  ability  to 
produce  its  output,  understood  in  terms  of  a  product  or  service  for  a  customer.  A 
performance  measurement  system  has  broader  managerial  implications  than  a  logistics 
metric. 

Caplice  and  Sheffi’s  criteria  for  performance  metrics  and  performance 
measurement  systems,  summarily  given  in  Table  4,  are  used  as  the  criteria  for  this 
evaluation  of  metrics  currently  used  by  MALS  to  measure  performance.  A  MALS  has  no 
standard  performance  measurement  system  and  thus  the  metric  criteria,  rather  than  the 
measurement  system  criteria,  underlie  the  evaluation  of  current  MALS  metrics. 


35 


Table  4.  Summary  of  Criteria 


Metric  Criteria 

Measurement  System 
Criteria 

Validity 

Robustness 

Usefulness 

Integration 

Economy 

Compatibility 

Level  of  Detail 

Behavioral  Soundness 

Comprehensive 

Causally  Oriented 
Vertically  Integrated 
Horizontally  Integrated 
Internally  Comparable 
Useful 

D.  MEASUREMENTS  AND  PERFORMANCE 

Performance  metrics  influence  the  actions  of  Marines  within  a  MALS  in 
particular  ways.  These  actions  contribute  to  a  certain  level  of  respective  performance  for 
the  MALS  as  a  whole.  This  section  examines  how  MALS  performance  is  impacted  by 
using  these  metrics.  More  specifically,  this  section  examines  the  resulting  effects  that 
flight  line  squadrons  experience  as  a  result  of  optimizing  respective  metrics.  Optimizing 
the  metric,  in  this  context,  refers  to  the  MALS  using  its  resources  to  improve  the  activity 
that  the  metric  measures. 

1.  Buffer  Management  Tool — Buffer  Status 

The  Buffer  Management  Tool  (BMT)  is  not  a  metric  in  and  of  itself.  It  does, 
however,  provide  a  wealth  of  analytical  data  that  is  useful  for  the  user.  Many  metrics  a 
MALS  uses  are  pulled  from  the  BMT.  One  metric  in  particular  that  the  BMT  produces 
that  is  used  frequently  (daily)  and  widely  within  the  MALS  maintenance  department  is 
Buffer  Status.  Another  is  TRR. 

The  Time  to  Reliably  Replenish  (TRR)  measures  the  amount  of  time  a  part  has 
spent  being  repaired  in  relation  to  the  expected  amount  of  time  to  repair  that  part.  This  is 
primarily  understood  in  this  context  as  a  maintenance  metric.  The  second  metric  is  a 
part’s  physical  buffer  status  and  it  refers  to  the  physical  inventory  level  of  that  part  on  the 
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ASO’s  supply  shelf.  These  metrics  are  often  reported  in  terms  of  how  many  black  or  red 
MAFs  there  are  currently. 

The  BMT  organizes  Maintenance  Action  Forms  (MAFs)  by  the  time  projected  to 
complete,  or  sign-off,  the  MAF.  Once  a  repairable  part  comes  to  the  MALS  for  repair,  a 
MAF  is  initiated.  This  MAF  documents  the  current  status  of  the  repair.  The  status  may 
range  from  being  In-Work  (IW)  to  Awaiting  Parts  (AWP).  A  myriad  of  other  status  codes 
exist  but  explanation  of  them  does  not  contribute  to  the  current  discussion. 

While  the  BMT  provides  a  wealth  of  information  regarding  the  status  of  each 
MAF  and  each  part  on  the  shelf,  certain  metrics  are  used  most  often  to  manage  the 
workload.  For  the  Supply  Department,  these  include  the  Requisition  TRR,  Consumable 
TRR,  Repairable  TRR,  and  Physical  Buffer  Status.  These  show  what  is  the  current  TRR 
versus  Design  TRR  of  parts  as  well  as  how  many  parts  have  a  physical  buffer  status  of 
red.  For  the  Maintenance  Department,  these  include  backlog  and  physical  buffer  impact 
of  inductions  as  well  as  Awaiting  Parts  (AWP)  Impact  reports.  These  show  how  their 
workload  is  impacting  the  physical  buffer  status  and  how  much  of  it  is  due  to 
maintenance  delays  rather  than  awaiting  parts  from  supply. 

The  purpose  of  the  BMT  is  to  help  maintenance  managers  better  prioritize  their 
workload.  According  to  the  NAE  CPI  Guidebook: 

BMT  was  initially  created  to  provide  Intermediate  Maintenance  Activities 
(I-Level)  the  insight  necessary  to  analyze  and  understand  the  demand 
placed  on  their  production  and  to  help  understand  their  ability  to  meet  that 
demand  while  revealing  the  time  it  actually  takes  them  to  reliably  repair 
(or  replenish)  the  asset  to  ready  for  issue  (RFI).  BMT  provides  the  ability 
to  see  constraints  in  the  production  system  and  to  make  decisions  where  to 
best  concentrate  CPI  activities  to  improve  production  capability  (Time  to 
Reliably  Replenish  or  TRR)  where  it  will  have  the  greatest  impact. 
(NAVAIR  6.7.2. 1,  2013,  p.  6-8) 

To  accomplish  this  task,  the  BMT  color  codes  each  MAF  according  to  the 
benchmarked  time  to  repair  that  part,  the  TRR.  The  NAE  CPI  Guidebook  (2013)  explains 
how  each  color  code  should  be  prioritized  (p.  6-14).  For  example,  any  MAF  with  a 
physical  buffer  status  of  red  should  be  worked  first  and  in  accordance  with  the  TRR.  If  a 
part  has  a  benchmark  of  30  days  for  repair,  it  is  expected  to  take  30  days  to  repair  that 
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part.  During  days  21-30,  the  MAF  has  a  color  code  of  red.  Red  MAFs  should  be  worked 
first,  in  a  “first  in,  first  served”  manner.  During  days  1 1-20,  the  MAF  has  a  color  code  of 
yellow.  If  a  MAF  is  yellow,  work  should  only  take  place  if  there  is  a  stoppage  on  all  red 
MAFs.  During  days  1-10,  this  MAF  has  a  color  code  of  green.  During  the  green  phase, 
work  on  this  repairable  should  only  commence  after  there  is  work  stoppage  on  all  yellow 
and  red  MAFs.  After  day  30,  the  MAF  has  a  color  code  of  black.  Black  MAFs,  the 
guidebook  explains,  should  only  be  worked  on  after  red,  yellow,  and  green  MAFs  are 
unavailable  for  further  work.  These  black  MAFs  should  be  prioritized  according  to  the 
ASO’s  physical  buffer  needs. 

The  reliability  of  the  benchmark,  or  the  Design  TRR,  for  each  repairable  is 
therefore  critical  to  how  beneficial  the  BMT  is.  The  process  a  MALS  generally  takes  to 
set  repair  time  benchmarks  is  quite  cumbersome  and  potentially  inaccurate.  To  set  a 
benchmark  for  a  part,  a  work  center  within  a  division  will  designate  a  “Design  Team”  to 
analyze  the  repair  process  and  determine  a  reasonable  estimation  of  the  repair  time  for 
that  part.  To  do  so,  it  chooses  the  repairable  part  and  track  the  time  of  every  step  required 
to  complete  the  repair  on  that  part.  That  repair  time  is  then  set  as  the  Design  TRR,  or 
benchmark,  for  that  repairable. 

Using  a  sample  size  of  one  cannot  produce  a  reliable  benchmark,  however.  In 
order  to  mitigate  this,  work  centers  will  generally  deduct  from  the  total  time  any  steps 
that  take  unusually  long.  The  evaluation  of  these  steps  as  potential  outliers  is  based  on  the 
knowledge  of  subject  matter  experts  on  the  Design  Team.  While  this  may  improve  the 
reliability  of  that  repair  time  as  representative  of  population,  it  still  falls  short  of 
providing  a  reliable  sample. 

To  further  cloud  the  problem,  parts  are  often  grouped  together  and  assigned  the 
benchmark  of  just  one  of  the  analyzed  parts.  This  is  done  because  an  individual  MALS 
has  hundreds,  potentially  thousands,  of  parts  that  it  repairs.  The  process  used  to 
benchmark  one  individual  repairable  is  time  consuming  and  obligates  valuable  man¬ 
hours.  To  do  this  repeatedly  for  each  repairable  would  severely  impair  the  maintenance 
department. 
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When  determining  parts  to  be  grouped  together  for  the  same  TRR  assignment, 
work  centers  will  often  choose  a  repairable  that  is  generally  representative  of  the  typical 
repair  process  for  parts  in  that  work  center.  The  benchmark  for  the  individual  part  is  then 
applied  as  the  benchmark  for  a  group  of  parts.  This  group  of  parts  could  be  all  parts  a 
work  center  repairs  or  groups  of  similar  parts  a  work  center  repairs. 

Grouping  parts  together  and  assigning  them  all  one  single  TRR  for  maintenance  is 
inherently  problematic,  however.  To  illustrate  this,  consider  a  notional  but  reasonably 
realistic  example  of  a  work  center  that  repairs  composite  main  and  tail  rotor  blades  for 
two  different  types  of  helicopters.  These  four  different  types  of  blades  have  different  part 
numbers  and  each  blade  has  a  multitude  of  types  of  repairs  that  may  be  required.  Each 
blade  can  only  be  assigned  one  TRR  and  this  TRR  does  not  change  with  different  types 
of  repairs.  Given  the  nature  of  composite  repair  and  parts  availability  for  Marine  Corps 
rotor  blades,  some  repairs  may  take  days  while  some  take  weeks. 

Prioritization  of  the  maintenance  workload  is  based  on  the  physical  buffer  status 
of  an  item  and  the  time  buffer  of  an  item,  its  TRR.  If  the  physical  buffer  status  on  two 
items  repaired  by  the  same  shop  is  equal,  the  one  with  the  shortest  TRR  will  take  priority 
if  shortest  processing  time  is  the  shop’s  dispatch  rule.  If  these  are  high  priority  parts,  then 
the  estimated  TRR  is  also  communicated  to  the  AMO,  ASO,  and  requisitioning  squadron 
for  other  planning  purposes.  If  the  repair  is  estimated  to  take  so  long  that  it  impairs 
readiness  at  the  squadron,  the  AMO  and  ASO  may  seek  an  alternate  solution  to  obtaining 
an  RFI  part.  In  the  example  of  repair  times  for  rotor  blade  repair,  cannibalization  of 
blades  or  lateral  support  requests  are  based  upon  the  maintenance  TRR  of  the  item. 

Using  four  different  rotor  blades  as  an  example,  notional  minimum  and  maximum 

total  delay  time  of  each  rotor  blade  are  demonstrated  in  Table  5.  The  90th  percentile  of 

each  blade’s  repair  time  distribution  is  its  TRR.  Specifically,  this  means  that  90%  of  the 

time,  this  blade  will  be  repaired  in  that  number  of  days  or  fewer.  Using  an  average  of 

repair  distributions  is  a  common  way  to  assign  multiple  parts  a  single  TRR.  In  this 

example,  21.58  days  is  the  average  of  all  the  blades’  repair  distributions.  The  cumulative 

density  function  (CDF)  indicates  the  total  probability  of  repairing  that  blade  in  a  time 

frame  less  than  or  equal  to  21.58  days  .  The  90th  percentile  and  averaged  TRR  of  all  parts 

39 


was  simulated  using  Crystal  Ball.  Beta  distributions  of  repair  times  for  each  blade  were 
assumed  and  100,000  trials  were  run  per  simulation. 


Table  5.  Repair  times  for  Notional  Parts,  TRR  at  90th  Percentile,  and  CDF  of  Each 

Part  if  Assigned  Averaged  90th  Percentile. 


Min  Repair 
(Days) 

Max  Repair 
(Days) 

90% 

(TRR) 

CDF  at 
21.58  days 

Blade  A 

3 

45 

31.58 

60.787 

Blade  B 

2 

25 

17.63 

99.045 

Blade  C 

1 

15 

10.5 

100.000 

Blade  D 

5 

65 

45.7 

31.341 

A  forecast  was  modeled  for  each  variable  individually  and  a  fifth  forecast  was 
modeled  of  the  average  of  the  four  variables.  The  90th  percentile  of  each  blade’s 
individual  forecast  is  demonstrated  in  Table  5.  The  90th  percentile  of  the  averaged 
forecast  was  21.58  days.  If  21.58  days  is  applied  to  each  blade  as  the  TRR,  managing  the 
maintenance  workload  based  on  TRR  becomes  problematic.  Two  of  the  blades  will 
consistently  exceed  its  TRR  and  two  will  consistently  finish  before  its  TRR.  One  can  see 
in  Figure  17  that  Blade  C  will  always  meet  its  TRR  while  Blade  D  will  only  meet  its  TRR 
31%  of  the  time.  If  any  one  of  the  TRRs  is  applied  to  all  blades,  rather  than  averaging  the 
forecasts,  the  results  are  no  more  helpful. 


100.000  Trials 


Frequency  View 


Figure  17.  Forecast  Blade  Repair  Time  Distributions  and  Average. 
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There  is  no  hard  requirement  for  work  centers  to  ensure  the  accuracy  of  their 
benchmarks.  According  to  the  NAMP,  the  MALS  ought  to  utilize  the  AIRSpeed  team  to 
“conduct  value  stream  mapping  and  analysis  to  benchmark  existing  processes  and 
illustrate  improvement  opportunities”  (Department  of  the  Navy,  2012,  p.  3-55).  This  is 
the  only  requirement  for  developing  benchmarks  for  repair  processes.  Ensuring  these 
benchmarks  are  updated  and  reasonable  is  only  implied  at  best  and  left  to  the 
responsibility  of  the  AIRSpeed  team. 

If  a  MALS  optimizes  the  buffer  status  metric  produced  by  the  BMT,  the  resulting 
goal  is  to  reduce  buffer  status  red  parts  on  the  supply  shelf.  In  order  to  reduce  buffer 
status  red  parts,  the  maintenance  department  will  prioritize  all  MAFs  for  parts  that  are 
red-coded  for  supply.  Red-coded  MAFs  will  always  be  worked  first,  followed  by  yellow 
and  then  green.  Any  MAFs  that  have  exceeded  their  TRR  are  coded  black  for 
maintenance  will  be  worked  into  the  red-yellow-green  rotation  of  MAF  prioritization. 

To  effectively  manage  the  workload  in  this  way,  Design  TRR  estimates  need  to  be 
reliable  and  specific  to  each  part.  Design  TRR  numbers  need  to  be  established  for  each 
part  individually  and  updated  as  processes  or  capabilities  change.  Otherwise  certain  parts 
may  always  exceed  their  TRR  and  be  consistently  coded  black.  A  maintenance  manager 
will  typically  follow  a  specific  dispatch  rule,  whether  explicitly  stated  or  not.  If  reducing 
red  physical  buffer  status  MAFs  is  the  priority  of  the  MALS,  a  shortest  processing  time 
dispatch  rule  may  be  adhered  to.  If  multiple  parts  are  assigned  a  non-specific  Design 
TRR,  a  part  may  be  inordinately  prioritized. 

As  holes  get  filled  on  the  supply  officer’s  shelf  (i.e.,  buffer  status  red  parts  are 
repaired  and  replenished),  the  squadron  has  an  increased  number  of  requisitions  filled 
immediately,  rather  than  be  coded  as  not-in-stock.  This  means  squadrons  are  both 
receiving  parts  and  resuming  work  on  aircraft  more  quickly.  The  MALS’  focus  on  filling 
holes  in  the  supply  shelf  positively  affects  its  fill  rate. 

2.  Time  to  Reliably  Replenish — TRR 

The  Time  to  Reliably  Replenish  metric  is  directly  related  to  the  previous 
discussion  regarding  the  Buffer  Management  Tool.  It  can  also  be  used  as  standalone 
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metric,  however.  According  to  the  NAMP,  TRR  is  generally  defined  as  “the  time,  in 
hours  or  days,  it  takes  once  a  part  is  pulled  from  the  supply  shelf  until  it  is  back  on  the 
supply  shelf  ready  for  issue”  (Department  of  the  Navy,  2012,  p.  A-79).  Specifically, 
however,  it  means  different  things  between  Maintenance  and  Supply. 

For  maintenance,  it  is  the  time  to  reliably  repair  a  part  that  it  receives.  For  supply, 
it  is  the  time  to  replenish  a  part  in  inventory  once  it  has  been  issued  to  a  flight  line 
squadron.  Broadly  speaking,  TRR  is  the  time  it  takes  to  order  a  part,  receive  it,  and  place 
it  on  the  shelf  as  ready  for  issue.  To  compute  the  TRR,  the  times  to  fill  historical 
requisitions  of  a  particular  item  are  measured.  The  90th  percentile  of  those  times  is  then 
applied  as  the  TRR  for  that  part  (Seagren,  2013). 

TRR  is  an  important  measurement  in  Marine  Corps  Aviation.  MALSP  MOD  is 
structured  around  the  ability  to  maintain  consistent  TRR  measurements  between  various 
nodes  within  the  logistic  support  framework.  MALSP  MOD  is  designed  to  reduce  the 
supply  footprint  in  forward  deployed  locations.  According  to  MARADMIN  175/16, 

MALSP  MOD  is  an  improved  expeditionary  AVLOG  concept  that  will 
provide  more  responsive,  scalable,  and  flexible  solutions  through 
properly- sized  spares  packages  while  achieving  near  real  time  global 
visibility  through  an  enhanced  information  technology  (IT)  and  nodal 
logistics  laydown  (NLL)  supply  chain  concept.  (Headquarters  Marine 
Corps,  2016,  p.l) 

In  order  to  ensure  a  certain  service  level  for  flight  line  squadrons  with  a  nodal, 
demand-pull  logistics  network,  the  TRR  between  nodes  in  the  supply  chain  need  to  be 
relatively  reliable  and  consistent.  If  the  TRR  between  nodes  contains  great  variance, 
inventory  buffer  levels  have  to  be  increased.  Maintaining  a  large  inventory  at 
the  different  nodes  is  one  of  the  very  situations  that  a  nodal  logistics  network  is  meant 
to  avoid. 

Optimizing  TRR  in  this  context  can  refer  to  either  reducing  the  time  it  takes  for 
maintenance  to  replenish  the  supply  shelf  or  the  time  it  takes  for  the  supply  chain  (often 
the  Defense  Logistics  Agency)  to  replenish  the  supply  shelf.  This  metric  does  not 
measure  the  time  to  fill  squadron  requisitions.  Optimizing  this  metric  can  be  approached 
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in  two  different  ways.  First,  it  can  mean  reducing  the  TRR  to  as  low  of  a  number  as 
possible.  Second,  it  can  mean  making  TRR  as  stable  as  possible. 

The  first  is  only  helpful  as  long  as  variation  is  minimal.  The  second  approach  is 
more  helpful,  even  if  the  TRR  is  larger.  This  is  because  the  supply  officer  can  maintain 
higher  service  levels  with  a  reliable  TRR  through  buffer  size  management  than  if  the 
TRR  is  unreliable  and  widely  varying.  High  variance  in  TRR  requires  larger  inventory 
buffers  to  maintain  satisfactory  service  levels  during  replenishment  times,  which  is  not 
always  possible  due  to  parts  availability  and  budgetary  constraints. 

To  demonstrate  this,  suppose  the  ASO  desires  a  service  level  of  90%,  demand  for 
part  X  is  constant  with  one  per  day,  and  lead  time  for  part  X  (i.e.,  TRR)  is  not  constant.  A 
service  level  of  90%  has  a  z  value  of  1.28.  To  determine  the  proper  amount  of  safety 
stock  ( SS ),  the  z  value  of  the  service  level  (z)  is  multiplied  by  the  standard  deviation  of 
demand  during  lead  time  ( oL )  (Chase,  Jacobs,  &  Aquilano,  2004,  p.  557).  Because 
demand  is  constant  in  this  example,  the  standard  deviation  of  demand  during  lead  time 
(DDLT)  will  only  increase  as  the  standard  deviation  of  lead  time  (i.e.,  variation) 
increases. 

Table  6  summarily  presents  two  scenarios  that  portray  the  two  ways  in  which 
TRR  can  be  optimized,  reducing  lead  time  and  reducing  variation  in  lead  time.  In 
Scenario  A,  AVLOG  reduces  mean  TRR  without  also  reducing  the  variability  of  TRR.  In 
this  example,  TRR  might  be  reduced  to  four  days  and  the  standard  deviation  of  that 
replenishment  time  may  also  be  four  days.  To  maintain  a  service  level  of  90%,  a  safety 
stock  of  approximately  five  items  needs  to  be  held  on  the  supply  shelf. 

Suppose  in  Scenario  B,  AVLOG  chooses  to  reduce  variability  of  TRR,  rather  than 
the  mean  TRR.  In  doing  so,  the  resulting  safety  stock  requirement  is  quite  different.  If 
TRR  is  eight  days  but  variability  of  the  replenishment  time  is  two  days,  approximately 
two  and  a  half  items  are  needed  as  safety  stock  on  the  supply  shelf  to  maintain  a  90% 
service  level.  The  TRR  might  be  twice  as  much  in  this  scenario  but  variability  of  that  lead 
time  is  half,  resulting  in  half  the  safety  stock  required. 
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Table  6.  Safety  Stock  Requirements  with  Different  TRR  Variability 


Scenario  A 

Scenario  B 

Demand  During  Lead  Time  (1/day) 

4 

8 

Standard  Deviation  of  Lead  Time  (oL) 

4 

2 

Service  level  desired  (z) 

90.00% 

90.00% 

Safety  Stock  (SS  -  z  *  oL) 

5.13 

2.56 

Optimizing  TRR  through  reduced  variation  can  therefore  be  more  beneficial  than 
reducing  mean  TRR  but  allowing  increased  variation  in  replenishment  times. 
Understanding  the  implications  of  variance  in  TRR  is  therefore  essential  for  proper  buffer 
size  management.  If  buffer  sizes  are  managed  appropriately,  TRR  itself  does  not 
necessarily  have  any  positive  or  negative  impact  on  squadrons.  The  squadron  is  only 
negatively  affected  if  they  order  a  part  that  is  not  in  stock.  Whether  a  part’s  TRR  is 
60  days  or  10  days,  the  squadron  is  only  affected  if  the  ASO’s  buffer  is  not  sufficient  to 
cover  that  transportation  time. 

3.  Beyond  Capability  of  Maintenance 

The  maintenance  department  occasionally  encounters  situations  where  they  have 
a  part  that  they  are  unable  to  repair.  When  this  situation  arises,  the  MAF  receives  an 
action  code  of  Beyond  Capability  of  Maintenance  (BCM).  There  are  various  reasons  a 
MALS  may  not  be  able  to  perform  maintenance  on  a  part.  Examples  include  lack  of 
parts,  lack  of  repair  equipment,  or  beyond  authorized  repair  depth.  The  BCM  codes, 
therefore,  have  accompanying  identifiers  to  describe  why  the  MALS  is  unable  to  perform 
the  repair.  These  codes  are  identified  in  Table  7. 


Table  7.  BCM  Codes 


BCM  1 

Repair  Not 
Authorized 

BCM  4 

Lack  of  Parts 

BCM  7 

Beyond  Authorized 
Repair  Depth 

BCM  2 

Lack  of  Equipment, 
Tools,  or  Facilities 

BCM  5 

Fails  Check  and  Test 

BCM  8 

Administrative 

BCM  3 

Lack  of  Technical 
Skills 

BCM  6 

Lack  of  Technical 
Data 

BCM  9 

Condemned 
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Parts  being  coded  BCM  are  of  concern  due  to  the  general  higher  cost  of 
conducting  repair  at  a  higher  level  of  repair,  including  the  Depot  level  or  manufacturer.  It 
is  generally  cheaper  to  repair  a  part  at  the  lowest  level  possible,  with  the  flight  line 
squadron  being  the  lowest,  followed  by  the  intermediate  level  MALS.  The  maintenance 
department  and  supply  department  are  both  interested  in  BCM  rates,  albeit  for  different 
reasons.  The  supply  department  is  primarily  interested  in  BCM  4  codes  because  that  is 
the  code  for  which  they  have  the  most  influence.  The  maintenance  department,  however, 
is  concerned  with  all  BCM  codes. 

High  BCM  rates  may  indicate  that  either  the  maintenance  or  supply  department  is 
unable  to  perform  its  job  adequately.  A  maintenance  department  reporting  a  BCM  3  does 
not  reflect  positively  on  the  maintainers  within  that  MALS.  If  a  MALS  must  BCM  a 
repairable,  the  BCM  7  code  is  the  one  preferred.  This  is  because  it  indicates  that  they 
have  sufficiently  done  all  they  are  authorized  to  do  and  are  required  to  submit  the  part  for 
a  higher  level  of  repair. 

The  BCM  metric  incentivizes  cost-reduction  through  in-house  repair  of  parts. 
Focusing  on  keeping  low  BCM  rates,  however,  may  diminish  MALS  support  to  flight 
line  squadrons.  There  are  often  long  periods  of  troubleshooting  and  repair  before  an  item 
receives  a  BCM  code  (with  the  exception  of  BCM  1  and  most  BCM  7  codes).  If  the  goal 
is  to  minimize  BCM  codes,  then  those  periods  of  troubleshooting  and  repair  will  be 
further  extended  as  maintainers  continue  to  attempt  successful  repair.  These  longer  repair 
times  could  potentially  result  in  longer  wait  times  for  the  flight  line  squadron  who 
requires  the  part. 

Optimizing  BCM  rates  means  to  reduce  the  number  of  BCMs  that  occur, 
primarily  BCM  4  and  BCM  7.  BCM  7  rates  can  be  reduced  through  BCM  interdiction, 
when  a  MALS  receives  authorization  to  conduct  maintenance  themselves  or  a  local 
artisan  that  is  not  typical  of  the  I-Level.  BCM  4  rates  can  be  reduced  through  parts 
acquisition  and  inventory  management. 

BCM  rates  only  positively  affect  the  flight  line  squadron  if  it  results  in  a 
ready-for-issue  part  being  issued  to  the  flight  line  squadron  more  quickly  than  had  the 
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part  been  sent  off  for  repair  and  a  ready-for-issue  part  been  requisitioned.  Low  or  high 
BCM  rates  themselves  do  not  necessarily  affect  fill  rates  for  flight  line  squadrons.  As 
long  as  a  MALS  has  spare  parts  on  the  supply  shelf  for  the  squadron,  BCM  rates  are 
irrelevant  (in  terms  of  flight  line  squadron  support). 

4.  Supply  Effectiveness  or  Perfect  Order  Fulfillment 

Supply  effectiveness,  or  fill  rate,  is  primarily  a  supply  metric,  measuring  what 
percentage  of  demands  are  filled  immediately  upon  requisition.  Items  that  are  not  in  stock 
or  not  carried  in  the  supply  system  detract  from  the  total  percentage.  If  a  requisitioned 
item  is  in  stock,  then  that  completed  requisition  will  contribute  to  the  total  percentage. 
This  metric  is  typically  described  as  net  supply  effectiveness  and  gross  supply 
effectiveness.  Net  supply  effectiveness  considers  only  items  that  are  carried  while  gross 
supply  effectiveness  considers  all  requisitions.  Balestreri  and  McDoniel  (2002)  define 
perfect  order  fulfillment  in  much  the  same  way  as  supply  effectiveness,  though  without 
net  and  gross  distinctions. 

This  metric  is  beneficial  for  several  reasons.  First,  it  is  easily  calculated.  Second, 
high  fill  rates  have  corresponding  tangible  effects.  Third,  it  most  easily  discussed  and 
communicated.  There  can  be,  however,  negative  aspects  of  this  metric.  Primarily,  this 
metric  can  treat  all  requisitions  as  equal,  requisitions  that  render  an  aircraft  non-mission 
capable  (coded  AKO)  or  partial-mission  capable  (coded  AK7).  In  order  to  capture  this 
disparity,  the  ASO  must  filter  for  requisitions  that  are  coded  AKO  and  AK7.  Further 
filtering  between  consumables  and  repairables  also  improves  the  helpfulness  of  this 
metric,  as  this  delineates  between  who  may  be  responsible  for  filling  a  hole  on  a  shelf 
(i.e.,  maintenance  or  supply). 

Another  negative  aspect  of  this  metric  is  the  absence  of  any  time  dimension 
consideration.  An  AKO  requisition  that  is  not  immediately  filled  will  degrade  the  supply 
effectiveness  measure.  If  that  requisition  takes  10  or  40  days  to  fill,  however,  is  not  taken 
into  account.  These  two  scenarios  have  very  different  second  order  effects  on  a  squadron. 
Dispatch  rules  that  properly  prioritize  these  outstanding  requisitions  by  examining  the 
time  dimension  of  these  requisitions  will  best  impact  the  squadron. 
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Optimizing  the  supply  effectiveness  measure  means  ensuring  that  when  a  part  is 
ordered,  it  is  in  stock.  This  often  equates  to  minimizing  holes  on  the  supply  shelf.  Priority 
should  be  given  to  holes  on  the  shelf  whose  respective  parts  are  ordered  the  most 
frequently,  however.  The  original  MALSP  model  focused  on  optimizing  fill  rate,  which 
resulted  in  greater  range  and  depth  on  the  supply  shelf.  The  end  result  of  such  focus 
resulted  in  large  inventories  forward  deployed. 

Supply  effectiveness  connects  the  MALS  very  directly  to  the  squadron  and  high 
supply  effectiveness  rates  enable  the  squadron  to  best  work  towards  improving  flight  line 
readiness.  The  effects  of  improved  supply  effectiveness  are  easily  seen.  It  results  in  flight 
line  squadrons  receiving  parts  more  often,  that  is,  having  less  not-in-stock  coded 
requisitions. 

Focusing  on  supply  effectiveness  as  a  single  metric,  however,  places  the  burden 
primarily  on  supply.  Maintenance  is  not  able  to  directly  influence  this  fill  rate  without 
receiving  direction  from  Supply.  While  maintenance  does  replenish  the  supply  shelf, 
supply  is  still  the  first  point  of  contact  in  answering  for  low  supply  effectiveness 
performance.  Supply  effectiveness  is  therefore  a  reasonable  measure  for  the  ASO  to 
examine  but  less  useful  for  the  MALS  as  an  entity. 

5.  IMA  Effectiveness 

An  Expeditious  Repair  (EXREP)  is  when  an  intermediate  maintenance  activity 
(IMA)  receives  a  non-RFI  repairable  part  from  a  squadron  and,  because  that  part  is 
rendering  an  aircraft  NMCS  or  PMCS,  needs  to  be  repaired  expeditiously.  An  EXREP  is 
defined  by  the  NAMP  in  this  way: 

The  processing  for  repair  of  NIS  or  NC  components  (repairable  or 
consumable).  These  components  must  be  in  support  of,  or  related  to,  an 
NMCS  or  PMCS,  situation.  This  processing  is  accomplished  by  the 
immediate  removal  of  the  component  from  the  aircraft,  expedited  delivery 
and  induction  for  repair,  and  the  earliest  return  to  RFI  status  for  supply 
issue  under  the  standard  material  issue  priority  system.  (Department  of  the 
Navy,  2012,  p.  A-25) 

In  a  similar  way  that  net  or  gross  supply  effectiveness  is  used  as  a  performance 

metric  for  the  supply  department,  the  maintenance  department  may  use  EXREPs  as  a 
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basis  for  measuring  performance.  This  metric  is  called  “IMA  Effectiveness”  and  is 
computed  by  dividing  the  number  of  EXREPs  the  IMA  receives  and  how  many  of  those 
are  repaired  and  reissued  as  RFI.  An  EXREP  occurs  because  there  is  not  an  asset  on  the 
Supply  Officer’s  shelf  that  can  be  immediately  issued  as  RFI  or  the  asset  is  not  carried. 

Optimizing  this  metric  in  the  maintenance  department  will  lead  to  the  immediate 
needs  of  the  squadron  being  met  as  long  as  the  parts  are  repairable  at  the  IMA  level. 
Because  the  parts  are  immediately  necessary  to  help  resolve  and  NMCS  or  PMCS 
discrepancy,  quick  turnaround  on  EXREPs  can  more  directly  be  linked  to  improved 
aircraft  readiness  than  working  on  MAFs  that  fill  yellow  physical  buffer  status  items  on 
the  supply  shelf. 

EXREPs,  however,  are  indicative  of  holes  on  the  supply  shelf.  A  perfectly 
optimized  supply  chain  may  render  EXREPs  nonexistent.  While  maintenance  will  do 
well  to  ensure  EXREPs  are  repaired  and  reissued  quickly,  the  MALS  could  potentially 
avoid  EXREPs  if  resources  were  effectively  allocated  towards  physical  buffer  status  red 
reductions,  forecasting,  and  inventory  management. 

To  achieve  a  high  IMA  Effectiveness  ratio,  a  MALS  must  repair  and  reissue 
EXREPs  within  the  same  month  they  are  issued,  or  reduce  the  number  of  EXREP 
occurrences.  The  former  is  accomplished  through  expedited  repair  processes  by 
maintenance,  the  latter  through  physical  buffer  status  red  reduction  by  maintenance  and 
supply  jointly.  Reducing  EXREP  occurrences  more  positively  affects  the  squadron  than 
expedited  repairs  because  this  metric  is  measured  in  terms  of  EXREPs  per  month,  not 
time  to  complete  an  EXREP.  If  an  IMA  receives  an  EXREP  on  the  first  day  of  the  month, 
the  MALS  will  receive  the  same  measure  for  IMA  effectiveness  whether  they  take 
28  days  or  two  days  to  reissue  that  item. 

6.  Supply  Response  Time 

Supply  Response  Time  (SRT)  is  a  fairly  direct  metric  and  focuses  specifically  on 
the  time  between  when  a  part  that  is  on-station  is  requisitioned  and  when  the  squadron 
receives  the  part.  High  priority  items,  for  example,  have  a  target  supply  response  time  for 
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part  delivery  of  one  hour.  This  measurement  is  generally  provided  as  an  average  of  all 
delivery  times  or  as  a  percentile. 

Supply,  however,  does  not  have  total  control  of  the  factors  that  affect  SRT. 
Misaligned  schedules  of  the  MALS  and  the  flight  line  squadron  can  render  it  impossible 
to  deliver  parts  at  certain  times.  A  flight  line  squadron’s  operating  schedule  is  more 
dynamic  than  a  MALS  operating  schedule.  For  example,  if  a  flight  line  squadron  is 
operating  on  a  limited  schedule  due  to  some  personnel  being  gone  for  training 
detachments,  there  may  not  be  personnel  at  the  squadron  to  accept  any  delivered  parts. 
This  negatively  impacts  supply’s  SRT. 

Optimizing  SRT  in  its  current  construct  (the  positive  measure  of  how  many 
requisitions  are  delivered  in  the  target  time)  may  not  have  any  remarkable  effects  on 
flight  line  squadrons.  SRT  is  a  positive  measurement  and  thus  is  indicative  only  of 
deliveries  made  successfully  within  the  target  time.  In  this  manner,  it  is  a  function  of 
parts  already  on  the  supply  shelf. 

An  SRT  of  90%  parts  delivered  within  the  target  time  is  misleading  regarding  the 
level  of  service  provided  to  the  squadron.  This  is  because  the  10%  of  parts  that  were  not 
delivered  within  the  target  time  are  unaccounted  for  in  this  metric.  Furthermore,  a  portion 
of  that  10%  may  be  just  outside  of  the  target  SRT.  Delivering  a  part  in  four  hours,  when 
its  target  time  was  one  hour,  will  not  have  any  significant  effect  on  aircraft  readiness. 
Measuring  only  the  on-station  time  of  a  requisition  is  therefore  only  marginally 
beneficial. 

7.  Supply  Chain  Response  Time 

The  Supply  Chain  Response  Time  (SCRT)  metric,  as  suggested  by  Balestreri  and 
McDoniel  (2002),  seeks  to  resolve  SRT’s  shortcomings  in  regard  to  accounting  only  for 
on-station  time  of  a  requisition.  They  define  on-station  supply  response  time  as  logistics 
response  time  and  and  the  off-station  time  of  a  requisition  as  customer  wait  time.  These 
two  measurements  are  combined  in  the  SCRT.  This  is  helpful  because  supply  response 
time  examines  only  a  portion  of  the  logistics  system’s  activity. 
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If  SCRT  is  made  a  function  of  parts  that  were  not  delivered  in  their  target  delivery 
time  frame,  then  the  metric  becomes  much  more  useful  and  beneficial  for  both  the  MALS 
and  the  flight  line  squadron.  By  capturing  this  time  as  a  negative  measurement  (rather 
than  the  positive  measurement  of  how  many  items  were  delivered  in  the  target  time 
frame),  the  potential  impact  of  delayed  delivery  can  be  better  quantified  and 
communicated.  To  do  so,  the  MALS  would  measure  only  the  parts  that  missed  their 
target  delivery  time. 

Categorizing  those  off-station  requisitions  according  to  their  physical  buffer 
status  would  further  improve  this  metric.  SRT  construed  in  this  way  would  be  a  negative 
measurement.  As  a  negative  measurement,  it  is  much  more  indicative  of  the  quality  of 
support  a  MALS  provides  the  squadron. 

SCRT  better  affects  flight  line  squadrons  if  taken  as  a  negative  measurement 
rather  than  a  positive  measurement.  Focusing  improvement  efforts  on  reducing  excessive 
delivery  time  results  in  the  flight  line  receiving  critical  parts  more  quickly.  Information  of 
this  sort  will  help  the  ASO  allocate  his  resources  more  effectively  to  improve  support  to 
the  flight  line. 


8.  MALS  Effectiveness 

MALS  Effectiveness  is  a  type  of  performance  measurement  system  used  by 
several  MALS  seeking  to  provide  a  holistic  view  of  maintenance  and  supply  functions 
in  one  metric.  The  MALS  Effectiveness  metric  is  calculated  by  averaging  the 
supply  effectiveness  metric  (i.e.,  filled  requisitions)  and  the  IMA  effectiveness  metric 
(i.e.,  repaired  EXREPs).  If  a  MALS  supply  effectiveness  metric  is  80%  and  the  IMA 
effectiveness  metric  is  60%,  then  MALS  effectiveness  is  70%.  While  supply 
effectiveness  captures  a  large  amount  of  the  output  from  supply,  EXREPs  are  only  a 
small  part  of  the  output  from  maintenance. 

For  example,  if  a  MALS  has  800  requisitions  and  supply  fills  700  of  them,  its 
supply  effectiveness  is  87.5%.  If  it  has  30  EXREPs  and  reissues  15  and  BCM  codes  the 
other  15,  its  IMA  effectiveness  is  50%.  The  MALS  effectiveness  of  this  MALS  is 
therefore  68.75%.  If  supply  achieved  a  measurement  of  100%  (i.e.,  filling  the  remaining 
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100  requisitions),  MALS  effectiveness  would  only  increase  to  75%.  If  supply’s 
performance  remained  the  same,  maintenance  would  have  to  achieve  a  measurement  of 
62.5%  (i.e.,  repairing  only  four  more  EXREPs)  to  achieve  75%. 

Because  of  its  disproportionate  nature,  focus  on  improvement  to  this  metric  may 
not  greatly  increase  support  to  the  squadron.  To  increase  MALS  Effectiveness  in  the 
previous  example,  maintenance  may  only  need  to  RFI  between  five  and  ten  EXREPs. 
Five  to  10  parts  out  of  830  requisitions  and  EXREPs  will  benefit  the  squadron  but  not 
greatly. 

E.  MEASUREMENTS  AND  BEHAVIOR 

Measurement  systems  will  affect  the  behavior  of  those  being  measured.  It  has 
been  discovered,  however,  that  some  measurement  systems  will  have  unintended 
consequences  for  overall  performance.  A  result  may  be  “that  the  types  of  behavior 
rewarded  are  those  which  the  rewarder  is  trying  to  discourage,  while  the  behavior  is  not 
being  rewarded  at  all”  (Kerr,  1995,  p.  7).  Another  disparity  between  measurements  and 
behavior  can  occur  when  a  positive  behavior  is  rewarded  but  “provides  no  actual  benefit 
for  the  system”  (Doerr  &  Gue,  2012,  p.  726).  This  can  occur  when  a  dispatch  rule  or 
policy  is  internally  focused  and  does  not  directly  contribute  to  system  output. 

This  section  focuses  primarily  on  examining  current  primary  metrics  used  by  the 
MALS  and  how  they  can  influence  the  behavior  of  Marines  in  a  MALS.  Each  metric  has 
examined  in  detail  in  this  section  also  in  order  to  fully  understand  the  metrics  and  how 
they  are  calculated.  Further  emphasis  is  placed  on  the  behavior  that  each  metric 
encourages  within  the  MALS. 

1.  Buffer  Management  Tool — Buffer  Status 

The  focus  of  the  BMT  system  is  to  rightly  prioritize  maintenance  in  order  to 
ensure  the  supply  shelf  is  adequately  stocked.  By  establishing  a  reliable  benchmark  for 
repair  time  of  part,  more  reliable  predictions  can  be  made  regarding  repair  completion. 
This  is  helpful  for  both  the  ASO  and  the  squadron  that  requires  that  part.  The  ASO  is 
better  able  to  manage  his  inventory  and  buffer  size  for  that  part  in  order  to  ensure 
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continued  availability  of  that  part  for  the  squadron.  If  parts  take  longer  to  repair  than  the 
benchmark  then  the  ASO  may  not  have  the  part  when  the  squadron  needs  it.  If  the  repair 
is  repeatedly  faster  than  the  benchmark,  then  the  ASO  may  have  unnecessary  extra  parts 
on  the  shelf. 

Proper  use  of  the  BMT  will  incentivize  Marines  to  prioritize  MAFs  that,  once 
completed,  will  fill  empty  spots  on  the  ASO’s  supply  shelf.  The  supply  shelf  becomes  the 
customer  that  the  maintenance  department  serves.  This  does  not  always  directly  correlate 
to  the  greatest  needs  of  flight  line  squadrons,  however. 

Not  all  empty  shelf  spots  have  equal  importance.  The  frequency  and  quantity  at 
which  an  item  is  ordered  dictates  the  level  of  importance  it  demands.  An  item  historically 
requisitioned  once  a  year  will  have  a  supply  buffer  status  of  red  until  it  is  replenished.  An 
item  requisitioned  once  a  day  will  have  the  same  buffer  status.  The  buffer  status  color 
code  itself  does  not  indicate  which  item  ought  to  take  priority. 

Optimizing  BMT  can  have  positive  and  negative  effects  on  Marines’  behavior. 
Positively,  it  theoretically  encourages  Marines  to  expedite  repair  or  processing  of  the 
most  needed  parts — those  that  are  buffer  status  red.  Negatively,  increasing  the  range  of 
the  supply  shelf  (i.e.,  adding  parts  to  carry  in  inventory)  may  be  discouraged.  By 
increasing  the  number  of  parts  that  carried,  there  are  more  opportunities  to  have 
occurrences  of  buffer  status  red. 

Design  TRR  numbers  are  easily  manipulated  and  if  a  part  repeatedly  exceeds  its 
Design  TRR,  the  TRR  can  simply  be  increased  with  much  less  effort  than  attempting  to 
improve  the  repair  process.  If  BMT  is  optimized,  Design  TRR  will  have  to  be  accurate. 
Ensuring  these  numbers  are  accurate  is  manpower  intensive  (under  the  current  construct). 
Allocating  resources  to  ensure  Design  TRR  numbers  are  current  and  accurate  incurs  great 
opportunity  cost,  as  every  man-hour  spent  on  such  a  task  is  one  less  man-hour  spent  on 
repairing  parts. 
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2.  Time  to  Reliably  Replenish — TRR 

Focusing  on  TRR  as  a  primary  metric  for  a  MALS  may  result  in  increased 
inventory  levels  because  to  account  for  the  variation  of  supplier  or  maintainer  times 
(direct  factors  of  supply’s  TRR),  a  larger  buffer  of  spares  is  required.  With  increased 
inventory  levels  come  increased  costs.  This  was  the  model  of  MALSP  I  but  is  contra  to 
the  underlying  principle  of  MALSP  MOD. 

However,  it  does  increase  the  level  of  support  provided  to  supported  squadrons. 
Reduced  TRR  and  increased  inventory  results  in  squadrons  having  what  they  need  more 
quickly.  In  a  fleet  of  aging  aircraft,  however,  increasing  inventory  of  certain  dynamic 
components  can  be  expensive  at  best  and  infeasible  at  worst. 

In  order  to  optimize  TRR,  the  processes  for  forecasting  shortages  must  be 
improved.  Currently,  there  is  no  framework  for  forecasting  what  may  reach  buffer  status 
red  without  timely  replenishment.  An  algorithm  that  factors  historical  demand,  current 
buffer  status,  and  estimated  time  to  delivery  can  provide  a  rough  indication  of  whether  a 
part  will  arrive  before  it  is  requisitioned  by  a  squadron.  Any  parts  indicated  by  such  a 
calculation  could  then  be  opportunities  for  the  ASO  to  begin  expediting  before  they 
become  needed  for  down  aircraft. 

3.  Beyond  Capability  of  Maintenance 

Optimizing  BCM  generally  means  making  every  effort  to  repair  parts  in-house, 
also  called  BCM  interdiction.  A  likely  result  of  this  is  elongated  repair  times,  as  a  MALS 
spends  more  time  troubleshooting  a  problem  or  seeking  authorization  to  conduct  certain 
repair  procedures  normally  reserved  for  higher  levels  of  maintenance.  If  these  parts  are 
needed  by  the  squadron,  then  delivery  may  be  delayed. 

A  reasonable  MALS  will  not  conduct  BCM  interdiction  on  a  part  that  is  needed 
immediately.  However,  if  BCM  interdiction  begins  first  on  a  low  priority  part  and  then 
the  part  becomes  high  priority,  delivery  of  an  RFI  part  may  be  delayed  due  to  previously 
started  BCM  interdiction.  In  this  scenario,  BCM  interdiction  can  have  negative 
consequences. 
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BCM  interdiction  can  also  be  problematic  when  it  requires  resources  that  may 
otherwise  be  better  spent  on  other  maintenance.  Determined  Marines  may  spend  an 
inordinate  amount  of  time  trying  to  repair  a  part  when  it  could  be  more  efficiently 
repaired  elsewhere.  While  BCM-ing  a  part  may  be  more  expensive,  reduced  aircraft 
readiness  may  arguably  be  more  expensive  in  terms  of  opportunity  cost. 

4.  Supply  Effectiveness  or  Perfect  Order  Fulfillment 

Optimizing  supply  effectiveness  encourages  Marines  to  increase  inventory  buffers 
in  order  to  ensure  parts  are  available  when  requisitioned.  This  leads  to  increased 
inventory  and  holding  costs.  However,  because  supply  effectiveness  is  generally  reported 
in  terms  of  Net  or  Gross  terms,  increasing  the  range  of  the  supply  shelf  may  be 
discouraged. 

Supply  effectiveness  is  a  measurement  that  attempts  to  capture  the  output  of  a 
MALS  to  its  customer.  However,  similar  to  SRT,  it  does  not  adequately  capture  the 
negative  impact  of  the  requisitions  that  go  off-station  due  to  being  not-in-stock  or  not 
carried.  The  vast  majority  of  conversations  between  the  Maintenance  Officer,  Supply 
Officer,  and  squadron  Maintenance/Material  Control  Officers  revolve  around  those  off- 
station  parts. 

5.  IMA  Effectiveness 

The  IMA  Effectiveness  metric  positively  increases  by  repairing  and  reissuing 
EXREP  parts  to  the  squadron  in  the  same  month  that  they  are  received.  A  MALS  will 
receive  a  100%  IMA  Effectiveness  if  they  receive  and  reissue  one-of-one  EXREPs  or  30- 
of-30.  Reducing  EXREPs,  therefore,  does  not  decrease  IMA  Effectiveness. 

If  this  metric  is  used  as  a  focal  point  for  performance  measurement  then  all  of  a 
work  center’s  resources  will  be  dedicated  to  that  EXREP  when  it  is  received.  If  the 
EXREP  cannot  be  repaired  by  the  work  center,  then  that  part  receives  a  BCM  code  and  is 
sent  to  a  higher  level  of  maintenance.  A  MALS  generally  makes  every  effort  to  quickly 
return  EXREPs  to  the  squadron  or  quickly  BCM  the  item  in  order  to  requisition  an  RFI 
asset. 
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When  an  EXREP  is  received,  a  work  center’s  attention  is  turned  away  from  its 
current  task  to  repair  the  inducted  EXREP  item.  This  pulls  manpower  away  from 
maintenance  on  other  items,  potentially  critical  buffer  status  red  items.  This  is  the  only 
case  when  allocated  resources  to  an  EXREP  item  may  prove  problematic.  Even  so, 
EXREPs  are  generally  a  very  small  portion  of  an  IMA’s  total  MAF  count  and  therefore 
do  not  drastically  impair  progress  on  current  maintenance  being  conducted.  Because 
EXREPs  are  such  a  small  portion  of  an  IMA’s  workload,  using  them  as  a  metric  for 
effectiveness  falls  short  of  capturing  the  IMA’s  activity  as  a  whole.  Supply  effectiveness 
is  a  much  more  holistic  measure  of  the  supply  department  than  IMA  Effectiveness  is  of 
the  maintenance  department. 

6.  Supply  Response  Time 

Focusing  on  SRT  may  lead  to  expeditious  delivery  of  priority  parts  that  are  in 
stock.  As  requisitions  are  placed,  high  priority  parts  (determined  by  their  priority  code) 
will  be  delivered  first  because  they  have  the  shortest  time  requirement.  Lower  priority 
parts  have  longer  target  times  to  be  delivered. 

SRT  is  normally  calculated  as  a  percentage  of  requisitions  that  were  filled  within 
the  target  timeframe.  In  this  regard,  it  is  a  positive  measure  communicating  the  successes 
of  the  supply  department.  SRT  construed  in  this  manner  can  negatively  affect  supply 
personnel  behavior,  however. 

A  MALS  supply  department  may  generally  have  an  SRT  success  rate  of  90%. 
This  means  that  90%  of  all  requisitions  were  delivered  within  the  target  timeframe. 
Framing  the  metric  in  this  positive  manner  does  not  communicate  the  negative  affect  the 
remaining  10%  may  have  on  the  flight  line  squadrons.  A  positive  metric  of  90%  may 
create  a  culture  of  complacency  within  Marines  as  it  diverts  the  focus  from  improvement 
opportunities. 

If  the  metric  is  communicated  in  a  negative  sense,  it  will  focus  on  the  remaining 
10%  that  did  not  meet  the  target  time  frame.  Furthermore,  if  the  metric  is  communicated 
not  as  a  ratio  but  as  a  time,  then  the  information  is  much  more  meaningful.  Examining  a 
certain  percentile  of  delivery  times  of  the  10%  that  did  not  meet  the  target  time  frame  not 
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only  highlights  improvement  opportunities  but  better  communicates  how  the  flight  line 
squadrons  are  being  affected. 

7.  Supply  Chain  Response  Time 

SCRT  measures  the  total  time  on-  and  off-station  of  a  requisition.  If  the 
measurement  is  reported  negatively  (i.e.,  only  measuring  those  requisitions  that  missed 
their  target  delivery  time),  then  the  MALS  can  gain  a  better  understanding  of  how  the 
logistics  network  is  impacting  the  flight  line  squadron.  A  negative  measurement  better 
communicates  to  the  logistics  system  how  it  may  be  hindering  aircraft  readiness.  It  also 
more  clearly  identifies  where  process  improvements  can  be  made. 

In  the  same  way  that  SRT  reported  positively  may  create  a  sense  of  complacency 
among  Marines,  SCRT  may  do  the  same.  If  a  SCRT  success  rate  of  90%  is  reported,  then 
the  MALS  is  delivering  90%  of  requisitions  in  the  target  time.  However,  the  remaining 
10%  are  the  requisitions  that  are  immediately  affecting  the  flight  line.  Separating  the 
remaining  10%  into  high  and  low  priority  documents  can  better  focus  MALS’  efforts  on 
the  critical  few  parts  that  are  negatively  affecting  the  flight  line. 

Because  SRT  only  measures  on-station  time,  and  SCRT  combines  both  on-station 
and  off-station  time,  SCRT  is  much  more  helpful  as  a  measurement.  While  it  is  true  that  a 
MALS  does  not  fully  own  any  of  the  off-station  supply  chain  processes,  it  does  have  a 
great  deal  of  influence  on  them.  Expediting  requisitions,  negotiations  with  item 
managers,  and  trade-offs  with  other  MALS  are  all  ways  that  a  MALS  can  reduce  off- 
station  time  of  a  requisition. 

8.  MALS  Effectiveness 

Giving  equal  weights  to  supply  effectiveness  and  IMA  effectiveness,  the  two 
metrics  that  are  averaged  to  determine  MALS  effectiveness,  is  disproportionate.  A  much 
greater  amount  of  work  must  be  done  by  the  supply  department  than  the  maintenance 
department  to  increase  the  metric.  Only  a  small  portion  of  the  IMA’s  workload  is 
captured  by  this  metric.  Because  of  the  disproportionate  nature  of  this  metric,  optimizing 
this  metric  may  result  in  demoralizing  supply  department  Marines  due  to  the 
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disproportionate  amount  of  work  they  must  exert  to  increase  their  effectiveness  in 
comparison  with  the  maintenance  department. 

In  the  maintenance  department.  Marines’  behavior  may  not  change  very 
drastically.  EXREPs  are  a  small  portion  of  their  regular  workload  and  only  receive 
attention  when  they  are  received.  Once  received,  they  are  prioritized  above  the  regular 
workload  and  every  effort  is  made  to  repair  the  item  as  quickly  as  possible.  If  it  cannot  be 
repaired  in-house,  it  is  BCMed  and  is  sent  to  a  higher  level.  Once  either  of  these  events 
takes  place,  the  Marines  return  to  their  regular  workload.  Because  EXREPs  are  not  part 
of  their  regularly  managed  workload,  focusing  on  MALS  Effectiveness  will  not 
incentivize  greater  production  on  a  day-to-day  basis  for  work  centers. 

F.  CRITERIA  APPLIED 

To  evaluate  and  compare  the  discussed  metrics,  Caplice  and  Sheffi’s  (1994) 
evaluation  criteria  are  used  and  adapted  for  relevancy  to  the  MALS.  Table  8  summarizes 
the  criteria  and  provides  examples  of  the  types  of  questions  asked  regarding  each  metric. 
Each  metric  is  then  scored  according  to  these  criteria.  A  score  of  “5”  indicates  that  the 
criterion  describes  that  metric  very  well.  A  score  of  “1”  indicates  that  the  criterion  poorly 
describes  that  metric.  The  scores  are  then  averaged  for  a  total  score  of  that  metric. 


Table  8.  Evaluation  Criteria.  Adapted  from  Caplice  and  Sheffi  (1994). 


Validity 

Does  it  capture  the  events  and  activities  being  measured  and  controls 
for  external  factors? 

Robustness 

Is  it  interpreted  similarly  by  users?  It  is  comparable  between  MALS? 

Is  it  repeatable? 

Usefulness 

It  is  readily  understandable  by  decision  maker?  Does  it  provide  a 
guide  for  action  to  be  taken? 

Integration 

Does  it  include  all  relevant  aspects  of  the  process?  Does  it  promote 
coordination  across  functions? 

Economy 

Does  the  benefit  of  using  the  metric  outweigh  the  costs  of  data 
collection,  analysis,  reporting? 

Compatibility 

It  is  compatible  with  existing  information,  material,  and  systems  in 
the  organization? 

Level  of  Detail 

Does  it  provide  a  sufficient  degree  of  granularity  or  aggregation  for 
the  user? 

Behavioral  Soundness 

Does  it  minimize  incentives  for  counter-productive  acts  or  game¬ 
playing?  Is  it  presented  in  useful  form? 

57 


1. 


Buffer  Management  Tool 


The  BMT  captures  a  great  deal  of  data  regarding  maintenance  and  supply 
functions.  The  Physical  Buffer  Status  Red  metric  is  one  of  the  most  often  used  metrics 
produced  by  this  system.  The  data  is  easily  collected  and  the  metric  is  easily 
communicated.  It  quickly  focuses  maintenance  and  supply  leaders  on  critical  needs  on 
the  supply  shelf. 

However,  due  to  the  variety  of  squadrons  supported  by  different  MALS,  the 
number  of  physical  buffer  status  red  items  does  not  communicate  what  level  of  support  is 
provided  to  squadrons  in  comparable  terms.  Physical  buffer  status  red  also  does  not 
communicate  information  regarding  what  parts  are  currently  needed  on  the  flight  line.  An 
item  does  not  have  to  be  requisitioned  in  order  to  have  a  red  physical  buffer  status.  Table 
9  summarizes  the  evaluation  of  this  metric  according  to  the  proposed  metric  criteria. 


Table  9.  Metric  Criteria  and  BMT 


Metric  Criteria 

Buffer 
Status  Red 
(BMT) 

Validity 

5 

Robustness 

3 

Usefulness 

4 

Integration 

4 

Economy 

4 

Compatibility 

5 

Level  of  Detail 

5 

Behavioral  Soundness 

4 

Sum 

34 

2.  Time  to  Reliably  Replenish 

The  MALSP  MOD  nodal  logistics  laydown  framework  is  largely  built  upon  the 
TRR  concept  and  thus  this  metric  is  a  commonly  understood  idea  across  AVLOG 
organizations.  TRR  can  be  defined  in  both  maintenance  and  supply  terms.  As  a 
maintenance  concept,  it  is  founded  on  the  Design  TRR  of  parts.  The  Design  TRR  is  easy 
to  manipulate,  difficult  to  maintain  accuracy,  and  arbitrary  if  applied  to  multiple  parts 
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based  on  a  single  part’s  analyzed  TRR.  As  a  supply  measurement,  it  is  founded  on  the 
historical  replenishment  times  of  parts  through  the  supply  chain,  and  is  therefore  more 
reliable  and  useful. 

TRR  does  not  measure  the  output  of  a  MALS  and  does  not  reflect  the  quality  of 
support  a  MALS  provides  to  squadrons.  While  it  is  a  helpful  metric  for  a  sub-function  in 
AVLOG,  it  is  less  helpful  when  measuring  MALS  performance.  Table  10  summarizes  the 
evaluation  of  this  metric  according  to  the  proposed  metric  criteria. 


Table  10.  Metric  Criteria  and  TRR 


Metric  Criteria 

TRR 

Validity 

3 

Robustness 

3 

Usefulness 

3 

Integration 

3 

Economy 

2 

Compatibility 

5 

Level  of  Detail 

3 

Behavioral  Soundness 

2 

Sum 

24 

3.  Beyond  Capability  of  Maintenance 

BCM  is  a  metric  that  is  used  in  every  MALS  but  is  not  directly  comparable 
between  different  MALS.  This  is  due  to  the  different  repair  capabilities  and  support 
requirements  at  each  MALS.  For  example,  new  aircraft  may  have  more  parts  that  have  to 
be  automatically  BCM  coded  than  legacy  aircraft  parts  due  to  proprietary  information  of 
manufacturers.  It  is  easily  calculated  and  communicated,  however. 

BCM  accurately  captures  a  portion  of  MALS  activity  but  it  does  not  capture 
MALS  output.  Rather,  it  measures  when  maintenance  is  not  able  to  repair  a  part  in-house. 
Because  a  BCM  part  does  not  directly  affect  flight  line  squadron  support,  it  is  less  helpful 
in  measuring  MALS  performance.  Table  11  summarizes  the  evaluation  of  this  metric 
according  to  the  proposed  metric  criteria. 
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Table  11.  Metric  Criteria  and  BCM 


Metric  Criteria 

BCM 

Validity 

4 

Robustness 

3 

Usefulness 

4 

Integration 

2 

Economy 

3 

Compatibility 

5 

Level  of  Detail 

4 

Behavioral  Soundness 

3 

Sum 

28 

4.  Supply  Effectiveness  or  Fill  Rate 

Supply  Effectiveness  is  a  commonly  used  and  well  understood  metric.  It  is 
computed  easily  and  directly  measures  the  output  of  a  MALS.  It  is  comparable  between 
MALS  but  is  most  helpful  if  it  is  compared  between  T/M/S  at  each  MALS.  This  will 
identify  any  potential  systematic  support  issues  in  a  T/M/S  that  are  AVLOG  wide  issues. 

Presenting  this  as  a  positive  metric  (i.e.,  successes  rather  than  failures)  may 
discourage  process  improvement  as  this  metric  is  generally  quite  high.  Presented 
negatively  with  emphasis  on  NMCS  and  PMCS  parts,  however,  focuses  leaders’  attention 
on  the  critical  few  items  that  are  negatively  affecting  the  flight  line.  The  evaluation  of  this 
metric  according  to  the  proposed  metric  criteria  is  summarized  in  Table  12. 


Table  12.  Metric  Criteria  and  Supply  Effectiveness 


Metric  Criteria 

Supply  Effectiveness 

Validity 

4 

Robustness 

4 

Usefulness 

4 

Integration 

4 

Economy 

4 

Compatibility 

5 

Level  of  Detail 

4 

Behavioral  Soundness 

3 

Sum 

32 
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5. 


IMA  Effectiveness 


IMA  Effectiveness  only  captures  a  very  small  portion  of  IMA  activity.  This 
metric  is  not  used  by  all  MALS  and  is  therefore  not  currently  comparable  between 
MALS.  This  metric  is  a  factor  of  repair  capabilities  and  support  requirements  and 
therefore  is  further  limited  in  comparability  between  MALS. 

It  is  an  easily  calculated  and  understood  metric,  however,  and  may  help  the  AMO 
and  ASO  identify  process  improvement  areas.  Because  delivering  an  RFI  part  to  the 
squadron  is  the  goal,  it  must  be  the  focus  of  this  metric.  Therefore,  the  quickest  route 
(i.e.,  BCM  or  repair)  to  delivery  that  RFI  part  should  be  pursued.  If  this  metric  is 
optimized,  it  may  encourage  Marines  to  troubleshoot  parts  longer  than  it  would  take  to 
order  and  receive  a  part  once  BCM  coded.  The  evaluation  of  this  metric  according  to  the 
proposed  metric  criteria  is  shown  in  Table  13. 


Table  13.  Metric  Criteria  and  IMA  Effectiveness 


Metric  Criteria 

IMA 

Effectiveness 

Validity 

2 

Robustness 

3 

Usefulness 

3 

Integration 

2 

Economy 

3 

Compatibility 

4 

Level  of  Detail 

2 

Behavioral  Soundness 

3 

Sum 

22 

6.  Supply  Response  Time 

Supply  Response  Time  measures  delivery  times  against  target  times  that  are 
standardized  for  all  MALS.  These  target  times  are  only  applicable  to  requisitions  once 
they  are  on  station.  This  limits  the  scope  of  this  metric  as  it  only  captures  a  small  portion 
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of  the  supply  chain.  It  does  not  focus  leaders’  attention  on  areas  that  can  have  significant 
impact  on  the  quality  of  flight  line  support. 


Using  this  metric  does  not  contribute  greatly  to  improving  flight  line  support  if 
optimized.  While  this  is  the  only  portion  of  the  chain  that  MALS  completely  owns,  it  is 
not  the  only  portion  with  which  they  have  influence,  and  thus  this  metric  is  too  narrowly 
focused.  The  evaluation  of  this  metric  according  to  the  proposed  metric  criteria  is  given 
in  Table  14. 


Table  14.  Metric  Criteria  and  SRT 


Metric  Criteria 

SRT 

Validity 

3 

Robustness 

3 

Usefulness 

3 

Integration 

2 

Economy 

2 

Compatibility 

5 

Level  of  Detail 

3 

Behavioral  Soundness 

3 

Sum 

24 

7.  Supply  Chain  Response  Time 

Supply  Chain  Response  Time  is  a  more  holistic  measurement  of  the  wait  time  a 
squadron  faces  from  requisition  to  delivery.  Focusing  on  NMCS  and  PMCS  requisitions 
improves  the  usability  and  applicability  of  this  metric.  It  can  be  easily  compared  between 
MALS  though  it  should  be  subdivided  for  each  T/M/S  that  the  MALS  supports. 

This  metric  is  not  currently  used  among  the  MALS,  however.  Also,  it  accounts  for 
a  great  deal  of  time  that  is  not  controlled  by  the  MALS.  Using  this  metric  may  enable 
AVLOG  leadership  to  better  focus  process  improvement  efforts  on  systematic  issues.  The 
evaluation  of  this  metric  according  to  the  proposed  metric  criteria  is  shown  in  Table  15. 
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Table  15.  Metric  Criteria  and  SCRT 


Metric  Criteria 

SCRT 

Validity 

4 

Robustness 

3 

Usefulness 

5 

Integration 

4 

Economy 

4 

Compatibility 

5 

Level  of  Detail 

4 

Behavioral  Soundness 

4 

Sum 

33 

8.  MALS  Effectiveness 

MALS  Effectiveness  is  not  a  standard  metric  used  by  all  MALS.  While  it  captures 
the  majority  of  output  from  the  supply  department,  it  only  narrowly  captures  the  output 
from  the  maintenance  department  that  it  captures.  A  high  priority  on  EXREPs  is  standard 
among  MALS  and  thus  this  metric  can  be  easily  communicated  and  understood. 

Because  this  metric  is  an  average  of  two  other  metrics  (supply  effectiveness  and 
IMA  effectiveness),  the  result  is  equal  emphasis  on  each  metric.  This  is  misleading 
however  and  using  this  as  a  performance  measurement  may  encourage  counter¬ 
productive  behavior  among  Marines.  The  evaluation  of  this  metric  according  to  the 
proposed  metric  criteria  is  shown  in  Table  15. 


Table  16.  Metric  Criteria  and  MALS  Effectiveness 


Metric  Criteria 

MALS  Effectiveness 

Validity 

3 

Robustness 

2 

Usefulness 

2 

Integration 

2 

Economy 

2 

Compatibility 

4 

Level  of  Detail 

2 

Behavioral  Soundness 

2 

Sum 

19 
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G.  SINGLE,  MULTIPLE,  AND  WEIGHTED  METRICS 

A  single  logistic  metric  is  limited  in  the  information  it  can  provide  about  an 
organization’s  overall  performance  and  is  thus  insufficient  as  a  holistic  performance 
measurement  system.  A  MALS  has  many  individual  metrics  available  to  measure  its 
plethora  of  sub-functions.  These  combined  sub-functions  produce  a  corporate  output.  If  a 
single  metric  must  be  used  to  measure  the  performance  of  the  MALS,  it  should  be  the 
metric  that  captures  this  output  most  thoroughly  and  in  terms  that  are  indicative  of  the 
impact  a  MALS  has  on  the  flight  line. 

No  single  metric  currently  used,  however,  perfectly  captures  the  output  of  a 
MALS.  Deliveries  are  the  primary  and  majority  output  of  a  MALS  but  other  smaller 
outputs  exist.  To  capture  all  the  outputs  of  a  MALS,  a  performance  measurement  system 
consisting  of  multiple  metrics  must  be  used.  Metrics  that  capture  these  outputs  would  be 
reported  together  as  indicative  of  performance.  A  group  of  multiple  metrics  may  be  used, 
therefore,  as  a  performance  measurement  system.  Groups  of  metrics,  however,  can 
become  confusing  when  being  communicated.  Additionally,  if  every  output  of  the  MALS 
is  captured,  the  number  of  metrics  may  become  cumbersome. 

Multiple  metrics  may  then  be  combined  into  a  single  performance  metric.  This 
method  has  advantages  and  disadvantages.  It  captures  multiple  metrics  into  one  and  is 
therefore  more  easily  communicated  and  briefed.  However,  if  a  simple  average  is  taken, 
equal  weight  is  being  applied  to  all  sub-metrics.  This  problem  is  seen  in  the  MALS 
Effectiveness  metric. 

Applying  appropriate  weights  to  each  of  the  sub-metrics  that  are  reflective  of  the 
amount  of  output  or  importance  of  the  sub-function  that  is  measured  is  one  way  to  avoid 
this  flaw  of  averages.  The  Navy  Submarine  Fleet  Maintenance  program  (Submarine  Fleet 
Maintenance  Metric  Working  Group,  2013)  approaches  their  performance  measurement 
in  this  fashion.  This  method  scales  each  metric  according  to  the  proper  amount  of 
influence  or  importance  it  has  in  the  overall  system. 

A  system  of  appropriately  weighed  metrics  is  still  not  without  problems,  however. 
A  high  score  on  one  metric  may  be  negatively  skewed  by  a  very  low  score  on  another 
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metric.  When  this  is  the  case,  presenting  only  one  combined  metric  will  not  present  any 
indication  of  this  problem.  To  control  for  this,  each  sub-metric  may  be  assigned  a  goal 
and  a  lower  control  limit.  If  a  sub-function  is  drastically  underperforming  (i.e.,  it  is 
scoring  below  a  sufficient  level)  then  that  can  be  identified  through  the  use  of  a  lower 
control  limit.  An  overall  combined  performance  measurement  can  still  be  presented  and 
if  there  are  any  outliers  that  are  underperforming,  more  detail  may  be  provided. 

H.  PERFORMANCE  MEASUREMENT  SYSTEMS 

The  creation  of  a  performance  measurement  system  for  a  MALS  ought  to  abide 
by  the  six  criteria  proposed  by  Caplice  and  Sheffi  (1995),  summarized  in  Table  17.  Doing 
so  will  ensure  the  organization  is  able  to  make  proper  decisions  to  provide  the  best 
support  to  the  customer.  A  performance  measurement  system  views  the  organization  in  a 
holistic  manner;  a  sum  of  parts  rather  than  many  separated  processes. 


Table  17.  Measurement  System  Criteria.  Adapted  from  Caplice  and  Sheffi  (1995) 

Measurement  System  Criteria 

Comprehensive 
Causally  Oriented 
Vertically  Integrated 
Horizontally  Integrated 
Internally  Comparable 
Useful 


A  comprehensive  performance  measurement  system  for  the  MALS  will  include 
metrics  that  capture  maintenance  and  supply  department  outputs  as  well  as  customer 
service.  These  are  the  primary  stakeholders  in  a  MALS  and  thus  should  be  a  focus  of  a 
performance  measurement  system.  Not  including  stakeholder  perspective  in  a 
measurement  system  will  lead  to  actions  taken  that  may  negatively  affect  one  of  them. 

A  causally  oriented  performance  measurement  system  will  measure  activities  that 
directly  influence  MALS  performance.  Performance,  in  this  sense,  is  understood  as 
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current  and  future  performance  (Caplice  &  Sheffi,  1995).  Performance  for  a  MALS  is 
primarily  a  factor  of  RFI  parts  delivered  to  squadrons. 

A  performance  measurement  system  that  is  vertically  integrated  in  a  MALS  will 
clearly  communicate  the  goal  and  strategy  of  the  MALS  to  department,  division,  and 
work  center  leaders.  It  will  be  reflective  of  the  overall  goal  of  the  MALS.  Marines  will 
understand  how  their  role  contributes  to  improving  performance  because  performance  is 
clearly  defined. 

A  horizontally  integrated  performance  measurement  system  in  the  MALS  will 
include  “pertinent  activies,  functions,  and  departments  along  the  process”  (p.  63).  To  do 
so,  meaningful  and  causally  oriented  metrics  from  MALS’  major  sub-functions  need  to 
be  included.  This  results  in  greater  integration  and  cooperation  in  a  MALS. 

An  internally  comparable  performance  measurement  system  is  one  that  can 
demonstrate  how  trade-offs  can  be  made  among  processes  in  the  MALS  to  increase 
performance.  An  example  of  these  trade-offs  may  be  how  repairing  physical  buffer  status 
red  items  and  deliving  parts  may  affect  one  another.  If  the  measurement  system  is 
internally  comparable,  then  the  metrics  affect  and  are  affected  by  one  another. 

Lastly,  a  performance  measurement  system  ought  to  be  useful  to  MALS 
leadership.  It  should  be  “readily  understandable  by  the  decision  maker  and  provide  a  a 
guide  for  action  to  be  taken”  (p.  65).  If  it  is  too  complex,  the  meaning  of  the  metrics  will 
be  ignored.  If  the  metrics  do  not  seem  meaningful,  they  will  not  be  perceived  as 
beneficial  and  therefore  will  not  be  used. 

I.  MALS  AND  T/M/S 

No  MALS  is  alike  in  terms  of  the  T/M/S  aircraft  it  supports.  Because  of  these 
differences,  metrics  are  not  directly  comparable  MALS-to-MALS.  The  performance  of  a 
MALS  that  supports  multiple  H-l  squadrons  cannot  be  directly  compared  to  a  MALS  that 
supports  multiple  F-18  squadrons.  This  is  due  to  the  systematic  differences  of  supporting 
certain  aircraft.  Systematic  supply  issues  for  a  specific  T/M/S  can  skew  metrics  unfairly. 
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In  order  to  compare  different  MALS,  the  metrics  should  be  MALS  and  T/M/S 
specific.  MALS-24  in  Hawaii  should  have  different  metrics  for  H-l  squadrons  and  CH- 
53  squadrons,  for  example.  If  the  CH-53  fleet  is  encountering  systematic  supply  issues, 
the  quality  of  support  for  the  H- 1  squadron  will  not  be  negatively  impacted. 

A  MALS  and  T/M/S  specific  measurement  system  will  result  in  a  MALS 
reporting  measurement  for  each  T/M/S  they  support.  As  an  example,  MALS-24  will 
report  a  performance  metric  for  their  H-l  squadron,  their  CH-53  squadron,  and  their  V-22 
squadron.  When  this  delineation  is  made  in  the  performance  metric,  MALS  support  for 
H-l  aircraft  can  be  compared  across  all  MALS  that  support  H-l  aircraft.  From  a  systems 
perspective,  this  gives  AVLOG  leaders  an  opportunity  to  identify  where  resources  should 
be  allocated  to  ensure  flight  line  squadrons  have  what  they  need  to  succeed. 

J.  CHAPTER  SUMMARY 

Currently,  the  MALS  examines  itself  according  to  several  logistics  metrics.  No 
performance  measurement  system  is  currently  used  to  evaluate  the  MALS  performance 
as  a  whole  and  current  metrics  are  insufficient  to  appropriate  capture  the  output  of  the 
MALS.  A  proper  performance  measurement  system  must  be  a  product  of  both 
maintenance  and  supply  metrics  that  measure  functions  in  a  MALS.  Returning  to  Caplice 
and  Sheffi’s  work  (1995),  deliveries  to  customers  are  the  output  of  a  logistics  system  and 
thus  should  be  the  focal  point  of  a  logistics  performance  measurement  system. 

The  metrics  a  MALS  uses  now  adequately  convey  the  performance  of  sub¬ 
functions  within  the  organization.  Optimizing  each  metric  has  different  implications  for 
the  performance  of  the  organization  as  a  whole  (in  terms  of  support  for  flight  line 
squadrons)  and  for  the  behavior  of  Marines  in  the  MALS.  Choosing  any  one  of  these 
metrics  individually  to  be  used  as  the  measurement  system  for  the  MALS  can  have 
negative  second-  and  third-order  effects  on  the  support  a  MALS  provides  to  flight  line 
squadrons. 

A  MALS’s  extent  of  control  ceases  at  the  point  of  delivery  to  a  squadron.  The 
goal  of  the  MALS,  therefore,  cannot  extent  beyond  this  point.  Increased  aircraft 
readiness,  therefore,  cannot  be  the  goal  of  a  MALS.  The  MALSP  MOD  nodal  logistics 
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system  is  strongly  indicative  of  Marine  Aviation’s  intent  to  operate  in  a  lean,  efficient 
manner.  Increasing  inventory  levels  to  support  flight  line  squadrons  is  no  longer  part  of 
the  model  and  thus  emphasizing  metrics  that  encourage  that  type  of  behavior  is  to  be 
discouraged. 
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V.  CONCLUSION  AND  RECOMMENDATIONS 


A.  CONCLUSION 

The  MALS  has  many  metrics  available  that  measure  its  various  sub-functions, 
though  they  vary  in  degrees  of  usefulness.  For  a  metric  to  be  a  valuable  logistic 
measurement,  it  needs  to  meet  certain  criteria.  The  predominant  MALS  metrics  are 
evaluated  against  the  criteria  proposed  by  Caplice  and  Sheffi  (1994)  and  summarized  in 
Table  18. 


Table  18.  Summary  Comparison  of  Metrics  and  Evaluation  Criteria 


Metric 

Criteria 

Buffer 
Status  Red 
(BMT) 

TRR 

BCM 

Supply 

Effective¬ 

ness 

IMA 

Effective¬ 

ness 

SRT 

SCRT 

MALS 

Effective¬ 

ness 

Validity 

5 

3 

4 

4 

2 

3 

4 

3 

Robustness 

3 

3 

3 

4 

3 

3 

3 

2 

Usefulness 

4 

3 

4 

4 

3 

3 

5 

2 

Integration 

4 

3 

2 

4 

2 

2 

4 

2 

Economy 

4 

2 

3 

4 

3 

2 

4 

2 

Compatibility 

5 

5 

5 

5 

4 

5 

5 

4 

Level  of  Detail 

5 

3 

4 

4 

2 

3 

4 

2 

Behavioral 

Soundness 

4 

2 

3 

3 

3 

3 

4 

2 

Sum 

34 

24 

28 

32 

22 

24 

33 

19 

The  most  useful  metrics  (e.g.,  physical  buffer  status  red,  supply  chain  response 
time,  supply  effectiveness)  sufficiently  fulfill  their  purpose  for  measuring  their  respective 
function  within  the  MALS.  Even  these  metrics,  however,  largely  fall  short  in  measuring 
the  overall  output  of  the  MALS  and  thus  do  not  meet  the  standard  for  a  performance 
measurement  system.  For  a  performance  measurement  system  to  be  most  beneficial  to  the 
organization,  it  needs  to  meet  the  criteria  proposed  by  Caplice  and  Sheffi  (1995)  for  a 
performance  measurement  system.  Individual  metrics  are  unable  to  be  comprehensive  or 
horizontally  integrated  (1995.). 


69 


1.  Primary  Questions 

(1)  Of  the  metrics  that  MALS  currently  employs,  which  measures  (or 
combination  thereof)  most  appropriately  reflect  the  level  of  support  the 
MALS  provides  the  flight-line? 

Of  the  metrics  that  MALS  currently  employs,  Supply  Effectiveness  most 
appropriately  reflects  the  level  of  support  the  MALS  provides  the  flight-line.  This  metric 
most  comprehensively  measures  the  majority  output  of  the  MALS.  Of  the  current 
metrics,  it  best  captures  the  overall  function  of  the  MALS.  Most  sub-functions  within  the 
MALS  contribute  to  this  end  metric  in  some  manner. 

This  metric  is  not  necessarily  ideal,  however.  It  is  not  very  descriptive  in  terms  of 
what  drives  high  or  low  support.  In  order  to  know  how  to  improve  Supply  Effectiveness, 
leaders  must  examine  separate  metrics  individually  to  determine  the  errant  process.  This 
metric  gives  leaders  no  indication  of  where  to  find  the  underlying  problem. 

Furthermore,  this  metric  is  framed  positively,  is  inclusive  of  high-  and  low- 
priority  requisitions,  and  provides  no  time  domain.  By  constructing  the  metric  in  this 
manner,  no  indication  is  provided  about  the  true  level  of  support  a  MALS  is  providing.  If 
Supply  Effectiveness  is  90%,  it  may  be  understood  that  a  high  level  of  support  is  being 
provided.  If  that  90%,  however,  is  primarily  low  priority  consumable  requisitions  and  the 
remaining  10%  are  primarily  NMCS  requisitions  that  have  remained  off-station  for  30 
days,  then  the  true  level  of  support  may  be  actually  quite  low.  This  metric  lacks  sufficient 
detail  to  inform  leaders  about  the  true  level  of  support  provided  to  a  flight-line  squadron. 

(2)  Of  the  metrics  that  MALS  currently  employs,  which  measures  most 
appropriately  incentivizes  the  most  appropriate  behavior  to  support  the 
flight-line? 

Of  the  metrics  that  MALS  currently  employs.  Repairable  Physical  Buffer  Status 
Red  and  TRR-Supply  incentivize  the  most  appropriate  behavior  to  support  the  flight-line. 
Physical  buffer  status  red  incentivizes  maintenance  and  supply  to  both  focus  work  efforts 
on  those  items  which,  if  requisitioned  by  the  flight-line,  will  not  be  able  to  be  filled. 
Prioritizing  these  items  contributes  to  filling  these  holes  on  the  supply  shelf  so  they  are 
available  when  needed.  TRR-Supply  incentivizes  proper  forecasting,  expediting,  and 
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inventory  management.  A  reliable  TRR  metric  aids  in  proper  inventory  management  and 
prioritization  of  expedited  requisitions.  By  examining  TRR  more  closely,  supply  is  able 
to  adjust  inventory  levels  or  expedite  requisitions  appropriately  to  increase  supply 
effectiveness. 

(3)  Are  there  metrics  that  MALS  does  not  employ  that  should  be  adopted? 

Two  metrics  should  be  adopted  by  the  MALS.  First,  the  Supply  Chain  Response 
Time  metric  should  be  adopted  as  a  logistic  metric.  Second,  a  weighted  performance 
metric  should  be  adopted  as  an  overall  performance  system. 

First,  Supply  Chain  Response  Time  should  be  adopted  by  the  MALS,  specifically 
in  regard  to  NMCS  and  PMCS  requisitions  by  T/M/S  aircraft.  By  measuring  the 
squadron’s  total  on-station  and  off-station  wait  time  for  NMCS  and  PMCS  requisitions,  a 
stronger  signal  is  sent  to  the  entire  supply  chain  than  by  just  measuring  on-station  wait 
time.  While  it  is  true  that  the  MALS  only  controls  the  amount  of  time  a  part  is  on-station, 
it  is  also  true  that  a  MALS  has  influence  on  portions  of  the  supply  chain  which  it  does  not 
own. 

Capturing  this  time  domain  of  the  supply  chain  from  the  customer’s  perspective  is 
an  excellent  measure  of  customer  satisfaction.  Furthermore,  by  capturing  this  metric 
according  to  the  various  T/M/S  a  MALS  supports,  systematic  issues  are  more  easily 
identified.  A  MALS  may  have  a  low  average  for  SCRT  but,  unless  it  is  delineated  by 
T/M/S,  poor  support  for  a  particular  T/M/S  may  be  masked  due  to  a  misleading  average. 

Second,  a  weighted  performance  metric  should  be  adopted  as  a  performance 
measurement  system  for  each  T/M/S  a  MALS  supports.  This  is  helpful  in  unifying  the 
MALS  under  one  common  goal  that  is  specific  and  measureable.  The  primary 
components  of  this  performance  metric  should  be: 

•  Physical  Buffer  Status  Red  of  Repairables:  The  percentage  of  repairable 
items  carried  that  have  a  physical  buffer  status  of  red 

•  Supply  Chain  Response  Time:  The  percentage  of  NMCS  and  PMCS 
requisitions  that  went  off- station  with  an  SCRT  exceeding  a  certain 
threshold 
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•  Supply  Effectiveness:  The  percentage  of  NMCS  and  PMCS  requisitions 
that  went  off- station 

By  presenting  these  metrics  as  percentages,  they  can  more  easily  be  compared  to 
other  MALS  that  support  the  same  T/M/S.  These  metrics  provide  a  holistic  view  of  the 
output  of  a  MALS  and  provide  enough  detail  to  MALS  and  AVLOG  leadership  to 
identify  why  support  for  a  flight-line  squadron  may  be  poor.  Condensing  this 
performance  metric  to  these  three  individual  metrics  enables  easy  communication  of  the 
metric. 

While  these  metrics  do  not  completely  capture  the  output  from  a  MALS  and  all 
the  services  they  perform  for  squadrons,  they  do  summarize  the  majority  of  the  output 
from  maintenance  and  supply  departments.  Though  the  Repairable  Physical  Buffer  Status 
Red  metric  is  a  measure  of  the  health  of  the  supply  shelf,  maintenance  is  primarily 
responsible  for  replenishing  those  repairable  items.  The  SCRT  metric  captures  the  impact 
off-station  items  have  on  the  flight-line.  The  Supply  Effectiveness  metric  captures  both 
consumable  and  repairable  requisitions  that  went  off-station. 

By  combining  these  metrics  into  one  measurement,  there  is  a  possibility  that  the 
end  measurement  may  fail  to  identify  problems  at  a  MALS  due  to  the  flaw  of  averages. 
Lor  example,  a  performance  measurement  goal  of  85%  can  be  reach  if  all  three 
measurement  score  85%.  It  can  also  be  reached  by  averaging  scores  of  90%,  95%,  and 
70%.  By  reporting  only  the  end  average,  the  lower  70%  metric  remains  unidentified.  To 
counteract  this,  each  measurement  should  have  an  appropriate  goal  to  define  what 
success  is  and  lower  control  limit  to  define  what  failure  is.  If  a  measurement  falls  below 
the  lower  control  limit,  it  ought  to  be  reported  separately  in  order  to  identify  the  area  for 
potential  troubleshooting. 

To  be  meaningful,  the  SCRT  metric  needs  to  be  measured  in  regard  to  a  certain 
threshold.  A  threshold  of  three  days,  for  example,  measures  the  number  of  requisitions 
exceeding  three  days  in  customer  wait  time.  Capturing  this  time  domain  is  a  strong 
indicator  of  responsiveness  of  the  supply  chain. 
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2.  Secondary  Question 

(1)  To  what  extent  can  standardized  performance  measurements  be  used  to 
measure  flight-line  support  performance  between  various  MALS? 

In  order  to  measure  and  compare  flight-line  support  performance  between  various 
MALS,  the  measurements  need  to  first  be  T/M/S  specific  within  each  MALS.  This 
accounts  for  the  systematic  differences  between  different  fleets  of  aircraft.  Legacy 
aircraft,  upgraded  aircraft,  and  new  T/M/S  all  have  different  support  systems.  A  MALS 
may  have  very  limited  repair  capability  on  a  new  T/M/S  due  to  manufacturer  proprietary 
rights.  Conversely,  a  MALS  have  a  great  deal  of  repair  capability  on  decades-old  legacy 
aircraft. 

If  a  MALS  does  not  measure  its  performance  according  to  T/M/S,  then  that 
MALS  cannot  be  compared  with  other  MALS.  A  MALS  that  supports  primarily  H-l 
aircraft  cannot  be  directly  compared  with  a  MALS  that  supports  primarily  F-18  aircraft. 
The  types  of  repairs  required,  availability  of  parts,  and  mission  requirements  for  these 
different  aircraft  limit  the  ability  to  compare  their  supporting  MALS  on  like  terms. 

By  measuring  performance  according  to  T/M/S,  AVLOG  leadership  is  better  able 
to  identify  where  it  should  allocate  resources  for  process  improvements.  Systematic 
difficulties  in  the  CH-53  community,  for  example,  will  be  difficult  to  identify  if  those 
maintenance  or  supply  issues  are  not  captured  separately.  Poor  support  performance  for  a 
particular  T/M/S  may  be  disregarded  if  it  is  averaged  in  a  conglomerate  metric  with  all 
T/M/S. 

B.  RECOMMENDATIONS 

The  first  recommendation  is  to  include  a  time  domain  when  measuring  the 
performance  of  a  MALS.  A  time  domain  is  generally  absent  among  the  majority  of 
metrics  used  within  the  MALS.  Unfilled  requisitions  do  hinder  flight-line  squadrons  from 
receiving  the  parts  that  they  need,  but  the  true  impact  of  unfilled  requisitions  cannot  be 
understood  without  measuring  how  long  squadrons  waiting  to  receive  those  parts.  A 
requisition  for  an  NMCS  discrepancy  has  very  different  implications  for  a  squadron  if  it 
is  delivered  30  days  rather  than  three  days. 
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The  second  recommendation  is  to  ensure  metrics  are  T/M/S  specific.  Resources 
may  be  misallocated  if  MALS  are  compared  to  one  another  without  this  distinction.  By 
measuring  a  fleet  of  aircraft  more  holistically,  a  stronger  signal  is  sent  when  a  systematic 
problem  occurs. 

The  third  recommendation  is  to  not  abandon  logistic  metrics  of  internal  functions 
within  a  MALS  when  adopting  a  holistic  performance  metric  for  the  MALS.  The  internal 
logistic  metrics  are  the  primary  way  in  which  leaders  can  identify  why  the  MALS 
performance  measurement  may  be  low  or  what  is  causing  it  to  succeed.  Internally, 
departments  and  divisions  should  continue  to  measure  their  internal  processes.  These 
internal  metrics,  however,  need  to  be  understood  in  relation  to  the  MALS  performance 
metric  and  how  they  affect  one  another. 

Lastly,  further  research  is  recommended  in  two  areas.  First,  cross  validation  of  the 
scores  given  to  individual  metrics  in  this  study  should  be  conducted.  Subject  matter 
experts  from  the  AVLOG  community  ought  to  independently  score  the  metrics  and  then 
the  inter-rater  reliability  or  correlation  between  the  scores  can  be  measured.  Second, 
appropriate  weights  ought  to  be  determined  for  each  metric  used  in  the  final  performance 
metric.  AVLOG  leaders  may  determine  if  these  metrics  should  be  assigned  equal  weight 
or  if  any  of  them  should  receive  more  or  less  weight  for  final  computation.  Weights  could 
also  be  assigned  by  analyzing  the  level  of  influence  each  metric’s  performance  has  on 
total  MALS  output.  This  will  prevent  any  of  the  measurements  from  unnecessarily 
skewing  the  measurement  due  to  disproportionate  allocation  of  weight. 
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